Technical SEO, out of all the optimization “subsets“, is often overlooked in informative articles. Witness the fact that link building alone brings back over 1 million results (hacked query). For technical SEO, you get about 2,730. If you modify the results based on the past year, it drops down to a measly 198 (about 10 of those are reprints from OutSpoken Media). Sad, sad.
Does this lack of coverage mean technical SEO isn’t as important as, say, link building? We all know the answer to that; without a working, clean site as the foundation, an SEO campaign just doesn’t have as much viability. In fact, it’s beyond me why such an important part of optimization is buried under other aspects.
Let’s add one more article to that measly number, shall we?
Technical SEO ““ What Is It?
For those of you who don’t know, technical SEO is just what it sounds like ““ dealing with all the technical issues, errors and bugs that come along from creating a website. The bigger the site becomes, the more issues you can potentially have.
Example:
Your client wrote a blog two years ago that has remained relevant. Over the two years, they’ve referenced that particular post in several other posts. It’s gained links from outside sites, people have talked about it and so on.
Then, for whatever the reason, that original blog needs to be deleted. They now have all those inbound links pointing to a 404 (error not found) page instead of good content. They also have all those internal links pointed to that post over the intervening years ““ and those links are pointing to a 404. That is, unless they knew to make corrections…
Your job, as the optimizer, is to clean your client’s website up and get it ready for the bulk of the campaign. To do this, you have to find the errors.
Tools to Find the Technical Errors
Crawl errors, duplicate meta data (or no meta data), poorly formatted URLs, bad HTTP response codes, duplicate content, slow site speed, clean code, a poorly written robots file ““ these are all elements that technical SEO targets. Luckily, there are several tools to help you find these issues:
SEO Spider ““ Search News Central’s own Terry Van Horne recently wrote a review about Screaming Frog’s SEO Spider, with a short video walkthrough. As the article and site point out, this tool can crawl the entire site. It’s £99 per year for the full version; the free version crawls up to 500 URLs.
SEO Spider can help you find a variety of issues, all with a friendly UI, including things like:
- Client and server-side errors
- Redirects
- Duplicate content issues
- Meta data
- Meta robots
- Size
- Inlinks and Outlinks
- Anchor text
SEO Spider doesn’t do everything for you, but it offers a fantastic start.
Xenu’s Link Sleuth ““ A free program, Xenu’s Link Sleuth offers some of the things SEO Spider does, without the friendly UI (the developer calls it a “no-frills user interface). For crawling sites, however, it’s a hard working program.
Xenu’s Link Sleuth also offers a variety of information, although not as much as SEO Spider. The information includes:
- Status of the page
- Type of link (application, image, text, etc)
- Size of the page
- Meta data
- Page level
- Inlinks / Outlinks
- Type of server
- Error response
- Date of the page
The crawl limit is approximately 10,000 URLs before it freaks out and has a nervous break down, so make sure you don’t run it on the really big sites.
IIS SEO Toolkit ““ Created by Microsoft for (as usual) the tech geeks, the IIS SEO Toolkit is a fully featured crawler. It’s only available for Windows Vista/7, so XP users are out of luck.
IIS SEO Toolkit provides information such as:
- Missing and/or duplicate meta data
- Different URLs pointing to the same page (duplicate content issues)
- Canonical violations
- H1/H2 and content key term usage
- Redirect violations
- HTTP Response headers
- Broken links
- Notifications of irrelevant anchor text
There’s a good review for the SEO Toolkit at XEN SEO, and a summarization of features at SEO Gadget.
Other Error-Hunting Tools;
If a large program isn’t what you’re looking for and you just need a little something extra, there are a variety of “partial” tools, as well. Here are just a few:
-
Google Webmaster Tools ““ There’s a lag in GWT for updates, but that shouldn’t stop you from using it as an overview. You’ll get a handy dandy list of 404s, soft 404s and “unreachables”. Sure, it won’t show you 3xx or 5xx, but hey, it’s free and it’s a good program to start with if you’re just now delving into the technical side of the Force ““ erm, I mean SEO.
-
Site Spider, Chrome Extension ““ If you’re a Google nut and/or Chrome user, you can install the Site Spider extension. It’s a simple crawler, showing you the status of the pages it crawls. However, since it goes wherever you are, it’s best not to be logged in to the site when you run it (for example, if the client is on a WordPress site).
Just because you get a 404 doesn’t mean the page is actually missing. For that matter, you can get many erroneous errors due to how the server handles the pages, how the code was written and so on. Before making changes, it might be a good idea to check the server-side header responses:
-
Firefox Live HTTP Headers ““ If you’re a Firefox aficionado, this Mozilla extension is a good addition to your developer toolbox. Redirects, information on hosts, servers and cookies, and more can be found!
-
Chrome Browser ““ Chrome itself has a built in developer’s panel. While much of it is more useful to site developers than SEOs, it also offers a network view of the site. Reloading the page with the network view open gives you the ability to look at each file’s headers, content, cookies and the time it takes to load.
Site Speed Tools
On top of all the above, you also have to consider the speed of your client’s site. For this, you’ll have to have some kind of network crawler, not just a page crawler. How fast does it take to load your client’s site? If it takes forever, what part of the code is the culprit? A few tools to help you out:
-
Chrome Browser’s Network tool ““ If you have Chrome, the network view (mentioned above) is very handy for seeing which elements take the longest to run. As well, the developer’s panel allows you to run a technical audit of a page, which includes network utilization and web page performance. Web page performance offers a bunch of error and warning information/tips, such as:
- External CSS, JavaScript that can be combined
- Files available for gzip compression
- Leveraging browser and proxy caching
- Minimizing cookie size
- Specify image dimensions missing
- Optimizing the order of styles and scripts
- Pingdom ““ Pingdom is an awesome little tool to use if you don’t have Chrome or prefer a web-based application. The only problem with Pingdom is that it can, and will, time out on big files (of course, if it times out on a page, you already know you’re going to have to pinpoint the issue of size ““ big isn’t always better).
- Page Speed Online ““ If you don’t want to install an extension or program, or you don’t want to use Google Chrome, you can always use Page Speed Online. The good thing about this web-based application, vs. the one offered by Pingdom, is that it provides a list of recommendations to fix the issues. The recommendations are listed according to high, medium and low priority.
- YSlow ““ For Firefox users, YSlow also summarizes, displays statistics and offers suggestions. This extension is a favorite of a lot of developers.
Keep in mind that these are just a few available ““ you can search the Chrome web store, Firefox Addons, or even the plain old SERPs for more extensions, applications and programs that might help you. These are just a few that we’ve used or are currently using.
Conclusion
Obviously, this is a short list of things included in technical SEO, and of tools to do the work. If you plan to make technical SEO a major part of your services, I strongly recommend trying several programs out before deciding on the one you’re most comfortable with. In this way, you have a better chance of getting a program that meets your needs, rather than pigeon-holing yourself into something less.
If you have your own fav technical on-site tool we missed, feel free to post it in the comments
Hey, thanks for this great breakdown of useful tools. Just what I was looking for.