How SERPWoo Crawls Data - Behind The Scenes

Have you ever wondered what goes on behind the scenes of a rank tracking operation or how SERPWoo stays accurate year after year while competitors seem to "fall behind"? Today we'll spill the beans on some of our techniques.

Google updates rankings multiple times a day - a lot of times hourly.

There are multiple Google datacenters around the world, they aren't all in sync so when an update occurs or simply ranking changes a user that is querying a datacenter in Chicago Illinois for a keyword will get varying results when compared to a user running a search in Seattle Washington - that's without taking into consideration local results.

FYI, if you want the most accurate data within SERPWoo use the locality option and input a city or postal code location you would like to query keywords for within the Project Wizard.

Use Locality for Location data within the Project Wizard

So in situations where you make a query and a colleague makes a query from a different location or even on a different browser or device you can get different results for the same keyword. Even if you two are standing two feet apart. This is the nature of Google.

With SERPWoo we attempt to hone in on accuracy with a number of techniques - one of which we'll talk about today:

We run two sets of crawling engines that do not interact with each other at any level. The first engine query the top 10, 20, and 30 search results and The second engine queries the top 10, 20, 30, 40, 50 - 100 search results at a different time. Since the Top 10, 20, and 30 results are the most important (Page 1-3) for online reputation management as well as basic search engine optimization (let's be serious who is going past page 3? No one) we use this two engine technique to figure out accuracy better.

A large number of our competitors only query once and do not bother with accuracy. So overtime you can start seeing less and less accurate results with competitor's tools - and this is before taking location data into consideration.

No longer accurate competitors show wild results

At SERPWoo we crawl keywords multiple times a day and compare the organic results - between the two engines. If there are significant changes from the first 2 crawls of today then we'll recrawl the data multiple times, and only once we are confident that the results we are pulling are what at least 90% of the users who are querying that keyword are seeing only then do we update the "Quick Glance" Tab and send out an alert (Quick Glance = Traditional Rank Tracking).

As this process is happening in the background when you view the "SERP View" you will always see the latest - even if we don't have 100% confidence yet. So if we crawled a keyword at 3:00 AM, then again at 5:00 AM, then again at 8:00 AM - you will be seeing the 8AM data.

In most scenarios our system would be comparing the 3:00 AM & 5:00 AM data and if there is significant changes it will keep re-crawling the data throughout the day until it builds enough confidence to update the Quick Glance (rank tracking) aspect of SERPWoo.

So there are instances where in the SERP View you will see changes in real-time if we get newer data from a SERP crawl.

SERP View - See Top 20/30, and 100 search results from a single view

Only once we have enough confidence do we update the Quick Glance data and send out an alert, if one needed according to your alert rules.

For the PDF reporting from the Report Builder the system waits till has 100% confidence with all keywords in a project before sending out a PDF Report. That way the data your client gets accounts for what over 90% of users see when they are querying the search results from their respective location.

Over the years SERPWoo has shown to be the most accurate search tracker ontop of being the first and original SERP tracker - we monitor and display all Top 100 results for a majority of keywords, normal rank trackers simply show you your data, yet cost alot more when comparing keyword capacity alone; nevermind the Keyword Finder tool and additional tools and features which brings added value to you as the end customer.

What's really crazy is SERPWoo seems to be one of the last premium daily rank trackers. A lot of competitors that attempted to steal ideas from us slowly and quietly switched to rank checking "every 3 days", then "once a week", and there are some that check "once a month" - What the hell are you paying for then?

That's the problem when you attempt to be a jack-of-all-trades, you cannot master one.

We employ a number of other techniques to gain the competitive edge over the old school rank trackers and their ways. But that's a part of the secret sauce that will probably never see the light of day.

For anyone serious and needing to monitor search results closely for SEO, Local SEO, or Online Reputation Management SERPWoo will continue being the dominate force in the industry.

Soon we'll be launching 2 new tools that will once again blow the minds of the SEO world and highly disrupt the Online Reputation Management aspect of search engine marketing. We will be re-staking our claim to the throne.

Also don't forget to use the bonus code "PaidMeUpScotty" for an extra 20% increase in keyword capacity to your paid plan (can only be applied once per account).

So stay tuned! And

To The Top of the SERPs!

---

---