Testing Sites for SEO: Seeing what the Crawlers See

Guest post from Anny Solway of Themefuse

One of the biggest challenges to improving the search engine optimization on a website is actually seeing what the search engine crawlers see. This is absolutely necessary to have any real idea of where your site stands in terms of SEO. Various tools available can help you to do this.

There are also some real advantages in making certain that your site is as easy as possible for search engine crawlers to read. Doing so generally means that your site is very lean – don’t mistake this with overly simple – and that it loads very quickly. In the book as SEO Made Simple, Michael Fleischner points out that the Google algorithm does take load speed into account. According to his work, the sites that rank the best in Google for any given keywords tend to load more quickly than the competition competing for the same keywords.

There are plenty of tools available that you can utilize to detect whether or not you have a smooth running, fast site. If you do, you should enjoy some advantages over the competition.

Browser Check

Most webmasters are well aware of the fact that they need to check their website in all of the major browsers. Most of them are likely also aware that they only need to worry about browsers that are standards compliant, as off-the-wall browsers may display perfectly valid code incorrectly. Many premium WordPress themes, in fact, advertise the fact that they work in all standards compliant browsers. This, however, is not enough. You have to do one more test.

Lynx is a text-only browser that is currently used for testing websites. This text browser allows you to get a look at your website in the same form that search engine crawlers see it in. According to Google itself, this is a good way to see if any multimedia content or other advanced features on your site are preventing the search engine crawlers from being able to read your content. Lynx gives a close enough approximation of what a search engine crawler sees that you can identify problems and correct them before they have too much of a negative impact on your search engine optimization.

Code Check

The Worldwide Web Consortium – W3C – offers tools that you can utilize to validate your code. These tools are completely free and they make certain that there aren’t any errors in your code that could be causing trouble. For some advanced webmasters, this may seem somewhat remedial but it’s important to remember that mistakes in code oftentimes result from very simple human error, rather than any type of incompetence. For that reason, it’s a good idea to put your pride to rest for a moment and validate your code to make sure that you didn’t make any obvious mistakes that could affect the search engine optimization of your site.

Another element of your code you will want to attend to is making certain that your web server supports If-Modified-Since HTTP headers. When Google comes to crawl your site, these headers let Google know whether any of your content has been changed. It can make it much easier for them to keep the index for your site fresh without having to burn up your bandwidth while Google indexes your entire site once again.

As a final note, if you are using themes for the WordPress platform or any other content management system, check the link structure. Using plug-ins such as Pretty Link for WordPress and equivalents for other platforms can ensure that your links don’t turn into a mess of numbers and letters that have bad search engine optimization qualities.