Banner

Follow Along

RSS Feed Join Us on Twitter On Facebook

Get Engaged

Banner

Featured Article

Using Your Site Statistics: Getting a Bird’s Eye ViewUsing  Your Site Statistics: Getting a Bird’s Eye ViewHow well do you understand your website design? Ask yourself....Are the design, usability, and informational factors top...
Read More >>

Latest Comments

Latest Articles

How to Improve Conversions  Through A/B TestingHow to Improve Conversions...
What is A/B testing? A/B testing is a method of testing that allows one...
Read More >>
Dialing in Your Call to ActionDialing in Your Call to...
Throughout the course of any marketing campaign, it is necessary to count conversions as...
Read More >>
Evaluating Digital Marketing ROIEvaluating Digital Marketing ROI
Imagine investing $15,000 on a print-based ad campaign. Your company would want proof the...
Read More >>

Our Sponsors

Banner
Banner
Banner

Latest Search Videos

Join Us

Banner
Banner
Does Google Tools Data Really Add up?
Written by Barry Adams
Wednesday, 22 February 2012 14:01

Most of us have by now realised that the data Google gives us in its various webmaster and AdWords tools is, at most horrendously inaccurate. But just how inaccurate is it? I decided to gather some web analytics data on one of my client sites and compare it to relevant data from Google Webmaster Tools (GWT) and the AdWords Keyword Tool (GAWKT), and see exactly where the pain points are.

Now I'm not the greatest of Excel wizards, so I just stuck with a basic table and a couple of graphs that I feel get the point across quite well.

First of all I went to Google Webmaster Tools and looked at the Search Engine Optimization reports for that website, specifically the Queries report. There I found 10 first-page ranked non-branded keywords, and exported all the relevant metrics to an Excel sheet.

Then I used the site's web analytics package's (Google Analytics) Traffic Sources report to see how many visits were sent to the site on those 10 keywords via Google organic search.

I also ran the keywords through the Google AdWords Keyword Tool to get the global & local search volumes for these 10 keywords, and I checked each keyword in the SERPs to get its current Google rank (on a private browsing session using unpersonalised search). For nearly all of these keywords I've been tracking their rankings for months and I know they haven't fluctuated much - if at all - in the last 30 days.

 

Data Accuracy in Google's Toolset

Now that I had all this data, I started comparing it. First I compared the AdWords Keyword Tool search volume data to the actual visits that the site received, and mapped the most recent rank of the keywords as well. With this comparison I hoped to see how well search volume and rank correlate with site visits.

This graph doesn't tell us very much aside from the fact that the amount of visits the site received on a specific keyword had little correlation with the search volumes as reported by Google's keyword tool. Rankings combined with local search volumes do seem to show some correlation, but there's an apparent disconnect between a keyword's rank and the percentage of visits it sends to the site (traffic share) - something which I explored further in a second graph.

Because the first graph was so inconclusive, I decided to map the data from the Webmaster Tools queries report to the traffic share and ranking data that I'd gathered:

Some points emerged from this graph:

  • The rank as I registered it myself in the SERPs has a good correlation with the average rank reported by GWT, though aside from a single instance they always differed, with the actual rank usually higher than the average rank GWT reports.

  • Traffic share (site visits as a percentage of local search volume) and click-through rate as reported by GWT are vastly different, with the traffic share % nearly consistently higher - and usually by quite a margin. Only once were they acceptably close.

  • When trying to determine which data source is polluting the figures, we can easily filter out some of the more obvious ones. As it turns out the GWT data is relatively accurate: visits on the website and click figures in GWT roughly match up, though the site's own data is again usually a wee bit higher:

If we then map the GWT Impressions data against the Local Search Volume data as reported by the AdWords Keyword Tool, a different picture emerges:

Here we see that in every instance the impressions number differs from both the global and local search volumes, and often this difference is quite significant. Usually the impressions number from GWT is much higher, but in a few instances it's a bit lower than the AdWords Keyword Tool's search volumes.

This graph, combined with the problem we had reconciling the traffic share data (which is derived from local search volume), can only lead to one conclusion: the search volume data from Google's AdWords Keyword Tool is the primary polluter of these metrics. While the other data sources show an acceptable level of cohesion, the data from the Keyword Tool invariably fails to match up in any meaningful way.

One last thing that I found highly intriguing was that the actual rank of a keyword - as well as the average rank reported by GWT - and that keyword's clickthrough rate don't seem to correlate very well. We often work off of the assumption that the top ranked keyword gets the most clicks, but this data leads me to suspect that might not be an entirely accurate assumption:

The highest CTR's seem to be found with rankings slightly below the top organic slot.

Now this is of course a totally unscientific experiment based on a single website, a limited data set, and no control data whatsoever. So realistically we can't - and shouldn't - draw any reliable conclusions from this.

Nonetheless I wanted to share this with you, as what SEO blogging generally lacks is data. I thought I'd add a wee bit of actual data to the ongoing debates about SEO best practices, and specifically about data accuracy. Do with it what you will - I know I shall.


Barry Adams -

Barry is the Senior Digital Marketer for Search at Pierce Communications in Belfast , where he works for a wide range of clients in the UK and Ireland. Since starting his career in the mid-'90s as an intranet content manager, Barry has worked in a wide variety of positions including corporate webmaster, in-house SEO specialist for a large regional newspaper, and web consultant for SMEs.

You can also hook up via

Read More >>


More articles by this author

Two Great SEO Tips for Large-Scale Ecommerce SitesTwo Great SEO Tips for Large-Scale Ecommerce Sites
When it comes to large scale ecommerce sites, many of...
Read More >>
The Death of SEOThe Death of SEO
Yes, I'm going there. Bear with me, this is not...
Read More >>
Last Updated on Wednesday, 22 February 2012 14:09
 

Comments  

 
0 #1 Terry Van Horne 2012-02-22 14:45
"We often work off of the assumption that the top ranked keyword gets the most clicks, but this data leads me to suspect that might not be an entirely accurate assumption"

Agreed because images and news items have changed the SERP... so IMO what you saw could be an influence of Universal Search
Quote
 
 
0 #2 Jordan McClements 2012-02-22 14:59
The AdWords traffic estimator also sucks. See :-

http://www.ppcni.com/accurate-google-adwords-traffic-estimator/1461/

(Though I haven't bothered with pretty graphs like you. :-)
Quote
 
 
0 #3 Barry Adams 2012-02-22 15:35
@Terry: yeah for these keywords there are a lot of shopping results on almost every SERP, which does tend to have a negative impact on CTR - even for top rankings.

@Jordan: aww man, pretty graphs are teh awesome!!11!1eleven. You missed a trick there. ;)
Quote
 
 
0 #4 Jahnelle Pittman 2012-02-22 16:29
Hey Barry - great article, pretty graphs :D Did you ever compare the data you gathered to site server stats? If you've done the work, I'd love to see it!
Quote
 
 
0 #5 erikeric 2012-02-22 22:21
Great article, thanks for sharing your findings. This is something I've worried about a lot this year.

I was working with a website who had already been pitched by Google themselves for PPC. After a couple of phone calls with the Google reps, they let lose that the AW KWT is not accurate because it is usually behind by months. Although it can be worse, they said, as they then revealed that the AW KWT was almost a year behind in search volume data for keywords related to online dating. I told them that was crazy and they very cautiously admitted that if we needed accurate figures, it was best to get them straight from a Google rep because their internal tools are kept up to date. Problem is, can't get a rep unless you're a brand new AW advertiser. Doesn't help SEO keyword research at all, which is what I mostly do.

The tool's figures and the rep's figures were so off it wasn't even funny.
Quote
 
 
0 #6 Barry Adams 2012-02-23 11:18
@Jahnelle no I sort of assumed the Google Analytics data would be at least mostly accurate. Server log analysis has its own issues and to be honest I haven't looked at server logs in a site traffic analytics context in years.

@erikerik well at least Google admits their AWKT data is woeful. That doesn't help us any, mind you, but it's a start. Maybe one day, in a utopic future, we'll have accurate search volume data. (Yeah, right.)
Quote
 
 
0 #7 Tiggerito 2012-02-27 02:24
I find another reason for its apparent inaccuracy is in localised search results.

If a SERP position is only valid for within 5km of your location then you're not going to get anywhere near the local volume indicated by the KeyWord Tool.
Quote
 
 
0 #8 Elevatelocal 2012-02-27 09:46
Excellent post Barry!
Quote
 
 
0 #9 Clare 2012-04-20 15:12
certainly makes you think about their accuracy!
Quote
 

Add comment


Security code
Refresh

Getting Around the Site

Home - all the latest on SNC
SEO - our collection of SEO articles
Technical SEO - for the geeks
Latest News - latest news in search
Analytics - measure up and convert
RSS Rack - feeds from around the industry
Search - looking for something specific?
Authors - Author Login
SEO Training - Our sister site
Contact Us - get in touch with SNC

What's New?

All content and images copyright Search News Central 2014
SNC is a Verve Developments production, the Forensic SEO Specialists- where Gypsies roam.