Written on October 5, 2012 by  /  with no comments  /  in the SEO category.

Two SEO Practices That Are Still Important in this Penguin-Hit Industry

This guest post was written by Emma-Julie Fox who writes for Pitstop Media Inc, a top rated SEO company that provides services to businesses across North America. If you would like to invite the author to guest post on your blog please contact www.pitstopmedia.com.

Google’s Penguin update shook the foundations of the SEO industry; there is no doubt about it. The update caused a massive furor; so much so that many have predicted the death of SEO a few years down the line.

At the same time, several reputable companies and websites have made statements that are in stark contrast to the fears of SEO professionals worldwide. Search Engine Journal has even posted an article saying that the Google Penguin update is actually good for the industry.

Is the SEO industry going to die out in light of this update? Definitely not.

Is the Penguin update going to change things? Yes; for the better – that is, if you’ve been practicing stringent, white-hat methods.

What practices will remain the same? These are what we’re going to discuss in this article. We may only cover two of them but these are two of the primary components powering white-hat SEO campaigns commonly used by ethical SEO services.

Practice #1: Embed sitemap.xml and robots.txt files to all your websites.

What is a “sitemap”?

You cannot believe how many fledgling webmasters are still oblivious about what this is. A sitemap isn’t complicated; it’s simply an XML file that shows all the URLS found on your websites.

Why is it important?

Getting all your important pages indexed by Google is essential if you want to have good visibility on the SERPs. A sitemap functions as a guide for the Googlebot to rapidly index the pages on your site that you deem should be listed by Google. It also informs search engines about which pages are more important, the frequency that you update the pages, and the dates of your last updates.

As stated, a sitemap isn’t a complex affair. It simply is an XML file that shows all the URLS found on your website. The above is a sample sitemap from Sitemaps.org.

(Source: http://cdn.seomoz.org/img/upload/sitemap-code.png)

How do you create and embed a sitemap?

Google itself has recommended a lot of third-party sitemap generators. You can use these user-friendly tools to generate your own. Embedding a sitemap is simple; all you have to do is use one of the recommended sitemap generators (of which http://www.xml-sitemaps.com is the most commonly used), create your own sitemap, and upload it to your site using an FTP client (FileZilla for example) or by using your cPanel File Manager. (Take note though that if you’re using the sitemap for your primary domain the document root you should be uploading it to your public_html folder; if the sitemap is for a secondary (add-on) domain, it should be uploaded to the document root for that domain.)

Robots.txt files are even simpler; they are just text files that tell search engine spiders what parts of your websites are available to them. Here you can stipulate what parts of your websites are restricted.

Now you may be wondering why webmasters should restrict certain parts of their websites. The advantages might not be immediately apparent but robots.txt files are useful in preventing spiders from crawling pages that contain duplicate content and ones that you deem private. It’s also a good way of conserving bandwidth; restricting bots from crawling your images for example, would prevent them from showing up on the SERPs.

Basically a robots.txt file uses the following syntax:



The disallow line tells search engine spiders what parts of your sites are restricted. For example, you can put there /images/sample-image1.jpg, telling the spiders not to crawl a specific image labeled as “sample-image1.jpg.” You can even block an entire directory by inputting /sample-directory/, where “sample-directory” is the name of the directory you want to be restricted.

Take note that the first forward slash represents your entire website so if you want to block your entire website, you can simply input only the forward slash and nothing else in the “Disallow:” line. (The preceding sentence should serve as a caveat when tinkering with your robots.txt file; be warned that you might inadvertently restrict access to pages that you otherwise might want to be crawled! If you’re unsure of what you’re doing, you can visit this Google page about the topic or better yet, consult with a trustworthy SEO company.

They may not be physical ones but search engine robots are as single-minded as their sci-fi counterparts; make sure you know what you’re doing when you’re tinkering with robots.txt files.

(Source: http://cdn.seomoz.org/img/beginners/disallow-robot.png)

After creating your robots.txt file, put it in the topmost directory of your domain as search engine spiders won’t look further if they can’t find it in the root directory. Thus, for a website named “ExampleForThisArticle.com,” the path to the .txt file should be: http://exampleforthisarticle.com/robots.txt.

Practice #2: Use Google Analytics for all your websites.

Most people in the SEO industry know how critical it is for webmasters to have a means of identifying and measuring the influx of people who visit their sites. Nonetheless a substantial percentage of webmasters still do not use Google Analytics, arguably the best and one of the most reliable SEO analytics tool (seeing that it has been created by no other than the predominant search engine company).

The following list details why you should use Google Analytics. This free service – among other things – enables you to:

  • See what keywords are bringing in traffic. Getting this information enables you to realign your SEO efforts to whatever your target market is looking for.
  • See the number of unique visitors on a day-to-day basis. You can even tweak parameters, enabling you to quickly see emerging trends over specified timespans.
  • See where the visitors are coming from. Traffic sources are important; quickly determining at a glance where the majority of your traffic is coming from enables you to refocus your efforts on the traffic-generating avenues.

Google’s Penguin update might have altered the SEO landscape but there are SEO practices that still hold to be effective even to this day, even if they are as simple as installing Google Analytics, sitemaps, and robots.txt files. Use them.