Building Websites To Fund A Travel Bug
Website Development
robots.txt
I recently created a robots.txt file for this site and the yet-to-be-launched Niche Site 1. Graywolf’s video is a nice primer on the problem of search engines crawling your blog, finding too much duplicate content, and lowering your PR. He’s a fan of WordPress and specifically made the video for it.
There are a few nice pages for the nitty gritty details. First is the official Web Robots page. Second is the sample robots.txt file on the SEO page at WordPress. Third is the robots.txt analysis tool under Google’s Webmaster Tools. This is probably the tip of the iceberg but I found these very useful for beginners like us.
47
If you enjoyed this article, make sure you subscribe to our RSS feed!
Related articles:
- Matching Colors for Google AdSense
- Checklist for New WordPress Sites
- WordPress Plugins
- Blog spam
- Using Preview To Modify Wordpress Themes Without Breaking Live Sites
Discussion
No comments for “robots.txt”
Post a comment