’s Margin of Error

When we try to figure out the growth of a site, or the relative traffic between sites, we often turn to the only free public tools available for the job.

That would be and Both are known to be inaccurate, however, Alexa perhaps more notoriously so.

The reason for their inaccuracy is the method of their tracking. Alexa relies on traffic stats from their toolbar that users must install. So, it’s not a representative cross section of the internet. Compete incorporates data from ISPs, toolbar users, and opt-in panels. Despite their added efforts, I’ve found their numbers to also be way off.

Finally, there’s Quantcast — another traffic estimator service. I’ve personally found their estimates to be equally inaccurate. However, unique to their service is the ability for site owners to place actual Quantcast tracking tags on their site so their traffic is directly measured. Once they collect enough data, they will display (if you chose to opt in) the actual traffic stats for all to see.

What this means is we can actually compare actual traffic stats to Compete’s estimated traffic stats (which are also reported in uniques/month) to see what Compete’s margin of error can be across different sites.

Based on a small sample of websites (n=25), Compete’s estimates predict from 18.6% to 137% of a site’s actually measured traffic stats.

See this graph (click for larger):


Explanation of Columns

Worldwide Uniques
  Actually measured Worldwide traffic via Quantcast

U.S. Uniques
  Actually measured U.S. traffic via Quantcast

Compete U.S. Uniques
  Compete’s estimate based on their aggregate data.

% of Actual world
  How far off is Compete’s U.S. numbers compared to Actual Worldwide

% of Actual US
  How far off is Compete’s U.S. numbers compared to Actual U.S.

Now, to be fair, Compete only claims to offer estimates of U.S. monthly uniques, but I’ve included Worldwide uniques to point out how deceptive this can be for sites with a large international audience, such as Compete’s U.S. estimate only counts 3.13% of Hi5’s actual worldwide traffic. Even when comparing U.S. numbers, Compete still underestimates Hi5’s U.S. traffic by over 50%.

Worst Estimates (U.S.)

In our small sample size, the most underestimated traffic sites where – 18% of Actual U.S traffic – 22% of Actual U.S. traffic – 24.8% of Actual U.S. traffic – 24.94% of Actual U.S. traffic – 27.97% of Actual U.S. traffic – 32.08% of Actual U.S. traffic -35.16% of Actual U.S. traffic

Best Estimates (U.S.)

These sites’ estimates were not that far off, and in some cases overestimated actual numbers. – 76.2% of Actual U.S. traffic – 76.2% of Actual U.S. traffic – 81.67% of Actual U.S. traffic – 93.70% of Actual U.S. traffic – 111.18% of Actual U.S. traffic – 112.79% of Actual U.S. traffic – 137.01% of Actual U.S. traffic


I’m not sure if many conclusions can be drawn from this data alone, but it just shows that traffic stats estimates can be very deceptive. I was somewhat surprised that some sites’ estimates were actually greater than their actual stats.

This entry was posted in Business. Bookmark the permalink.

8 Responses to’s Margin of Error

  1. Outofscope says:

    Did You try to compare these figures with Alexa’s estimations?
    It seems to me that Alexa has more significant toolbar installation base.
    My site is in Alexa’s top 100k for a certain period, but neither Compete nor Quantcast have got any data regarding it.

  2. Arnold Kim says:

    Alexa doesn’t report in “uniques per month” so it’s not directly comparable.


  3. Get Smart says:

    this is the stupidest thing i have ever read. what is “actual” traffic. cookie counts from server logs? cookie counts notoriously undercount traffic. idiot.

  4. Arnold Kim says:

    actual traffic is measured based on javascript placed on the site itself. It’s the same method as Google Analytics. It’s probably based on unique IPs, not cookies.

    It’s as accurate a measure there is.


  5. Hi5 in Alexa says:

    While hi5 maintains its strong hold in Central America, it faces tough competition in Asia from many social networking sites. comes in first ten ranking in You can see ranking of Fortune Park Hotels Ltd in Alexa.

  6. Pingback: Normalkid:Arnold Kim » Blog Archive » Engadget vs Gizmodo Stats

  7. We monitor our visitors’ IP addresses, operating systems, city, state, country, time of visit, and more. We see visitors in real time. We know how many visitors we have. According to our website had 179 visitors. We actually had 3,103 unique visitors, which are documented. Obviously,’s margin of error is as much as 94%. What is the point of Compete?

  8. paketresor says:

    keep up the nice work on the site. I kinda like it! 🙂 Could use some more frequent updates, but i’m quite sure you got better things to do like we all do. =p

Comments are closed.