It is probably a well-known fact, but Google webmaster tools record errors when they crawl the website. Of course, first of all the website has to be submittet to Google. After that, the crawl errors can be accessed by selecting the website of interest and clicking Health. On the left side, Crawl Errors will be available.
I did not even know that this was available until I received an email from Google which informed me that there was an increase in server errors and provided me a link to review those errors.
Google Webmaster Tools
Looks like the website was generating about ~70 errors when crawled, and I did not know. When the number of errors increased to ~90, I got an email. The errors are listed and can be marked as fixed as I deal with the root cause.
Website Crawl Errors
Most errors were caused by my refactoring where I replaced direct access to the database with using the repository pattern, and did it carelessly.
Another reason was that I used a tool I found on the web to generate my website map and did not review it before uploading on the website. The map contained some links that were not supposed to be accessed directly. It is probably a good idea to review those links to verify they are really needed or can be replaced and if they are needed, remove them from the site map. I'll be tracking the errors from now on.
References
Webmaster toolsCrawl errors
Sitemap Generator
by Evgeny. Also posted on my website
No comments:
Post a Comment