So you’ve got a new website that you have just laboured over with a great deal of love and attention (and probably spent a few dollars too) and now that it’s live, probably the first thing you do is go to Google or DuckDuckGo and do a search for your website. Then, to your utter amazement (or frustration) it appears or in the latter case it does not appear in the search you’ve done. If it does appear it probably means that your web developers have done their job extremely well (with something called SEO which we will talk about later). If it didn’t appear (at least not on the several pages you scrolled through) why didn’t it?
Like many things in our daily lives, we rarely have to look under the covers or even care so much about how something works because we just take for granted it does, or it might be too complicated to understand so we don’t bother trying (like how do they launch a shuttle into space), or it is just irrelevant to us how it works as long as it does. Searching the Internet probably falls into the first or latter for us – it just works so why do I need to understand what it does. However, as a business owner who’s paid money to get a website built, you need to know a little bit of how you can get your website found, otherwise your investment just won’t pay off and you won’t get the customer’s or attention you are hoping for from your new site.
Let’s look at the process then of how your website gets noticed on Google and other search engines.
The first step is to tell the search engines that your website is ready to be indexed (the term often used is ‘crawled’). This is done by the website developers who add special information into your website (such as the robot.txt file) that tell search engines IF the website can be indexed and can also tell the search engines WHAT can be indexed (i.e. the whole site or a limited set of pages).
The second step is the search engines need to be notified that your website is ready for ‘crawling’ – in other words it is ready to be searched and indexed. This last step can be done many ways and we’ll explain some of those options later. Normally, however, once your domain name is registered, the top search engines like Bing, Google, Ask, etc. have ways to monitor new domain registrations and they will periodically look for websites for new domains. Alternatively, you can go to each of the search engines and tell the search engine your site is ready to be indexed (i.e. you submit your site to the search engine which we will cover in a later post).
Once the search engine knows about your site then it [your site] will be added to its list of millions of websites that must be crawled, reviewed, indexed and analyzed so that your website can be found.