• Austin SEO
    • TastyPlacement in the Press
    • Meet the Team
  • Blog
  • Services
    • SEO Services
      • WordPress SEO Service
      • Magento SEO Services
      • Conversion Rate Optimization
      • Why Google Certification Matters
    • PPC & Adwords
      • Adwords & PPC Management
      • Remarketing Services
      • Display Ad Management
      • Facebook Ad Management
      • Pinterest Ad Management
      • Google Ad Grants Management for Non-Profits
      • Adwords App Install Ad Management
      • Product Listing Ad Management
    • Analytics & Data
      • Analytics and Monitoring
      • Google Tag Manager Experts
      • Data Studio Development & Consulting
    • Social Media & Local Marketing
      • Social Media Marketing
      • Local SEO
    • Web Development
      • Mobile Website Design
      • WordPress Development
  • Case Studies
    • SEO Case Studies
      • SEO Case Study: We Beat Lowes, Then We Beat Home Depot
      • SEO Case Study: Total Domination in Houston for Medical Provider
    • Analytics Case Studies
      • Case Study: Updated Design Yields 43% Increase in Conversion Rate
      • Case Study: PPC Optimization Yields Tripled Conversion Rate
    • Social Media Case Studies
      • Social Media Case Study: Hundreds of New Customers From Core Facebook Campaign
  • Portfolios
    • Display Ad Portfolio
    • Design Portfolio
    • Infographic Portfolio
    • SEO Testimonials
  • Contact
    • New Customers: Get to Know Us
    • Customer Service & Support
    • Referral Program
    • SEO Training Seminars
    • Job: Paid Search/PPC/Adwords Analyst
    • Job: Local Digital Marketing Specialist
    • Job: SEO/Marketing Junior Analyst
    • Privacy Policy & Terms of Use
  • Menu Menu
From the Wordpress SEO book

Should You Disallow Old Link Structures With Robots.TXT?

July 7, 2011/in Our Book: SEO for Wordpress/by Michael David

Questions from Readers…

We’re getting great questions from readers of our book, WordPress 3.0 Search Engine Optimization. Today, Michael tackles a question sent in by Jeff of Houston, TX. Remember, send in those questions and feedback! We’re always  thrilled to help out our readers.

Hi Mr. David,

I’m sorry to contact you with such an insignificant matter, but I just got your book today and wanted to ask if you could clarify an issue that I have encountered. My site has been up for about 6 months and I had been using a permalink structure of /year/month/day/postname and I changed it to /category/postname. I also used Deans Permalink Migration plugin to add 301 redirects for published posts.

I want to use your Ultimate Robot.txt file to my site, but I’m wondering if I add the “Disallow: /2011/ ” directive to eliminate duplicate content in my archives, will it disallow my previous posts that had /2011/ in the old permalink structure? Any help or clarification on this issue would be very appreciated. Thank you for your time.

Jeff

Houston, TX

Jeff,

We love hearing from readers.

Yes, I believe that if you add the directive Disallow: /2011/ you will remove year archives from indexing, but also any post that uses the year in that position as part of its permalink structure. I tested it, and it appears to disallow the content.

You can test your robots.txt file by using Google Webmasters’ Crawler Access testing tool. The tool lets you test the text of a robots.txt file and compare it to a specific URL. The tool then tells you if your robots.txt file is allowing or blocking the URL. You can find the tool by logging into Google.com/webmasters and then selecting “Site Configuration” and then “Crawler Access” from the left menu. We didn’t cover this specific tip in WordPress 3.0 Search Engine Optimization, but we will implement it in a future edition of the book.

Now, but you say you’ve changed your permalink structure–that should solve the problem. In the case where a robots.txt entry would block regular blog posts from getting indexed when blocking year archives, the solution is clear: don’t block either. Just make sure your year archive is set to display excerpts of the posts, rather than the full text of the posts.

Michael

Buy the Book Today at Amazon

Tags: Google, HTML5, seo book, WordPress, WordPress SEO, wordpress themes, XHTML
Share this entry
  • Share on Facebook
  • Share on Twitter
You might also like
How to Fix WordPress Robots.txt after Googlebot Cannot Access JS and CSS Warning
Footer Links: Bad for Google Rankings, Bad for Clients
WordPress Stripping iFrame Elements? Here’s the Fix.
Are Site-wide H1 Tags in WordPress Good or Bad?
Pubcon Austin 2013 How to Get “Show Map of” Tag to Appear in Google Results
From the Wordpress SEO book SEO Master Class: Choosing a Keyword-Rich Domain Name

Tutorials & Case Studies

  • Analytics
  • Backlink Strategies
  • Case Studies
  • Infographics
  • Internet Marketing
  • Local Maps and Local Listings
  • Magento
  • Mobile SEO
  • Our Book: SEO for Wordpress
  • PPC
  • Programming & PHP
  • SEO
  • SEO Power Tools
  • SEO Resources
  • Social Media Marketing
  • Updates & News
  • Web Design
  • WordPress

Our Most Recent Tutorials & Case Studies

  • Test Results: How to Stop Google Re-Writing Your Title Tags in the SERPs
  • Pro Tip: Track Your Website Goals in Google Analytics (and More)
  • How to Connect Google DataStudio to MySQL Database (cPanel Flavor)
  • UTM Codes/Tags: A Quick Guide to Tagging Ads Like a Pro
  • Social Media Case Study: Hundreds of New Customers From Core Facebook Campaign

Search

Archives & Tutorial Categories

  • Analytics
  • Backlink Strategies
  • Case Studies
  • Infographics
  • Internet Marketing
  • Local Maps and Local Listings
  • Magento
  • Mobile SEO
  • Our Book: SEO for Wordpress
  • PPC
  • Programming & PHP
  • SEO
  • SEO Power Tools
  • SEO Resources
  • Social Media Marketing
  • Updates & News
  • Web Design
  • WordPress

Austin SEO Company, TastyPlacement

TastyPlacement
4150 Freidrich Ln Ste C
Austin, TX 78744
Tel: (512) 535-2492

Google Maps: Get Directions or Read Our Awesome Reviews

Quick Navigation: Our Most Important Pages

  • Austin SEO [Home]
  • WordPress SEO Service
  • PPC Management
  • Social Media Marketing
  • Analytics and Monitoring
  • Remarketing Experts
  • Conversion Rate Optimization
© Copyright - TastyPlacement. Made in Austin, Texas
  • Twitter
  • Facebook
Are Site-wide H1 Tags in WordPress Good or Bad? Free Local Listings for SEO
Scroll to top