How Search Engines Work, Going Deeper

One of the most frequently asked Questions I hear is . . . "Why can't my site be found on Google?'

Its not mystical, a search engine is software. The purpose if the software is to retrieve data, index data then respond with data to a query.

Retrieval Spiders

The retrieval is done by "spider software". You probably have heard the crawling analogy given to understand the data retrieval, but in reality, the spider software just requests information much like any browser you use to surf the web.

The oh-so-important difference between search engine spider software and our browser is that the spiders do not collect images, formatting, and flash. The spider is only interested in text, links and the URL from which the data comes. Spiders love links because they lead to more … you guessed it, text, links and URL's.

Understand the evil concept of the "link blockers" many web designers and "get it up quick" sites use and why it is so important not to block the spider from getting all the pages of a site. It now makes sense to use CSS. CSS detaches the display, the look and feel, from the data so that the spiders only see the data and don’t get confused by endless use of inline tags creating yet another block to retrieving your data.

And now you may understand why Fire Up uses both "text linking" and CSS as central components to web site design - it helps your success.

Indexing

The indexing software receives data from the spiders and makes sense of the text, links and URL's using a complex mathematical formula called an algorithm. The algorithm analyses all the information and assigns a scoring which allows the Query software judge how important your page is for a specific related search query.

Take a second to ponder just what this means to the ranking of your pages. If we knew what the keywords, the searches, people were using to find the data on our pages AND we made certain your pages were indexed highly for those searches; then with those searches, we can greatly increase the probability of your pages being shown as a search result.

So that means we can use the text data, links and URL's as a way of making sure we get a high "relevance score" for one more searches. That process is called “optimizing”.

Welcome to the fascinating world of Search Engine Optimization and all it can do for us!

Query Software

The query software is what you see in our browser search results. It is what shows you the data the spiders got and then uses the indexing software ranking to show you a list of results. Type in your search, hit search and the search engine will do its best to match your keywords with the best pages it can find in its indexing.

Starting at the End

So you see, the more you know about the keywords people are using for searching, the better you can relate your pages to those searches. AND, the more you design your web site based around searching, the more successful your web site will be.