8 Tips for creating spider friendly websites
Tuesday, April 25th, 2006Is your HTML code spider friendly?
This is the first question to ponder when evaluating a website for SEO. Search engine spiders have a limited amount of time to crawl a website, so it is absolutely important that their time is spent crawling your relevant content. Using common sense coding habits and creating clean code will usually facilitate this, but there are many other techniques that can make your site easier to index.
I have found the following techniques to be very effective when building a spider friendly website.
Do not overload your pages with graphics. Flash and java look great but they are invisible to the search engines. Having too much of either will only clutter your pages and use up valuable crawling time. Moderation is the key. It is possible to make a website that is both attractive and spider friendly. Just remember to put your java and CSS in external files.
Meta tags no longer carry the weight that they once had, but they are still worth writing. The meta description should get the most attention because it is used to describe your pages in the SERP’s, and each page should have a unique description otherwise it may be ignored by the spiders.
Title tags are of utmost importance. These tags tell the spider what your pages are about and they are also the first line in the SERP’s. Some types of websites (i.e.dynamic/e-commerce) may require reconfiguring (mod_rewrite) the server to generate correct title tags.
Spiders tend to read your pages just like we do, top to bottom and left to right. Knowing this, you should always put your most pertinent information at the top of your page. This is one reason for having keywords and phrases at the beginning of your text. Also relocating extraneous code will make room to move your text closer to the top.
Spiders love header tags (h1, h2, etc.), but make sure to use only one H1 tag per page or it may be viewed as spam. Also, be sure to define your header tags in the CSS to make your site more visually pleasing.
Use tableless designs (XHTML & CSS) to avoid hiding your content from the spiders. Tables have a tendency to bury content.
Use a robots.text file to hide unimportant content and scipts from spiders. This will make your coding much cleaner.
Spider simulators, like the one Summit Media offers, are available to test your code and determine whether or not it is spider friendly. I recommend using these to refine and trouble shoot your design.
These are basic but important coding techniques that every SEO should be familiar with. If you’ve been concentrating on link building and your rankings haven’t improved, maybe you should take a look at your HTML code. The web (code) that you have woven may be too tangled for even a spider to crawl.
I like to stress the importance of
Sometimes it’s the little, overlooked details that can really affect your online traffic.