How to accurately estimate Steam game sales

In this article, we'll explore several methods of estimating game sales on Steam. Mainly, we will look at the following estimation methods:

1. Review multiples (Boxleiter method)
2. Using the Steam's top seller rank to estimate revenue and sales.
3. Polling public steam profiles to estimate game ownership. Similar to how SteamSpy used to work before the Steam profile privacy change.
4. Using the number of concurrent players to estimate game's sales

At the end, we will come to an algorithm that estimates sales and revenue much more accurately than the review based approach.

To test the estimates, I collected this sample of around 120 games with public sales data and made a script that compares game sales to estimates on a given date. This is an unbiased sample that contains games of various sizes, genres and years of release. Due to high error margins, we will ignore games that sold less than 1000 copies in our tests.

Review multiples (Boxleiter method)

Let's start with the simplest method. Taking the number of reviews and multiplying it by a constant number. The median sales/review ratio is around 35. Here are the results after running a test:

That's relatively inaccurate. Less than half of games are within 30% error margin and 30% of games are off by more than 50%

Now, let's try to improve our method by using different multiples for different release dates, price, review scores, and the like. Something similar to what is described here and here.

This looks a little better, but is still pretty imprecise.

Estimating game sales using concurrent player count and average playtime estimates

Now let's try another method. We can add up the number of concurrent players for every hour and divide that by the average playtime estimate to estimate game's playerbase. Since number of concurrent players is reported as an exact number via steam API, accuracy of this method depends completely on the average playtime estimate. We can estimate average playtime from all public indicators like public profiles and game reviews. The more data points we can gather, the more accurate the estimate will be. Here are the results after running a test:

This is better than the review based approach, but is still not accurate enough. Also, for some games we do not have accurate historical concurrent player data, so this method cannot be applied to all games.

Using Steam's top seller rank to estimate sales

Let's try one more thing. Using the steam top seller rank to estimate game revenue and sales.

Top seller rank is a good indicator of how a game is doing. Steam's top seller lists are generated automatically based on all revenue sources for a game, including DLCs and in-game-transactions.

This is a double-edged sword, as it allows us to estimate revenue for free-to-play games but can disrupt unit sales estimations for paid games with in-app-purchases, so we have to be careful with that.

Here are the results of the test.

Polling public profiles

Finally, let's try polling public Steam profiles to estimate game ownership. This is the method used by SteamSpy prior to Steam's privacy policy change. Since the number of public profiles has been significantly reduced since Steam's policy change, instead of the 3-day rollback used by steamSpy, we will use a much larger rollback of 30 days to collect a sufficient sample. And then, we will use the number of reviews and the top seller rank to fine-tune the estimate.

This method seems to be the most accurate for now, however, the problem with this approach is that it doesn't really work for smaller games, the margin of error is simply too large. So for anything under 20,000 players, we'll have to rely on other estimation methods.

Aggregating estimates

Now let's aggregate all of the above methods and see what we will get.

We will use different weights for different estimation methods depending on the game (eg for smaller games, profile polls are given less weight)

Additionaly, I will use this 2018 leaked dataset to further adjust the estimates for older games.

Here are the results:

Now, this looks much better (than the review based approach we started with)!

Here is a detailed overview of the test results:

Estimating revenue

However, this does not tells us the whole story. We also want to know the revenue of the game, and sales and revenue do not have to be linearly correlated. Luckily, I made a small algorithm that factors in discounts when calculating revenue based on games price history profile. This should give us the right ball-park for the vast majority of games.

Unfortunately, for some games, the situation is not that simple. It can be hard to differentiate copies sold vs given away for free, and even harder to differentiate copies sold regularly and copies sold in bundles. Further more, it's pretty much impossible to know how the game has sold on 3rd-party sites.

We can use the ratio of reviews marked as purchased on Steam / Activated with a key, look for discrepancies in review ratios and playtime data, look for patterns in public Steam libraries that may indicate a game was purchased in a bundle, and use top seller rankings to help us deduce how many copies were sold directly through steam. However, there is currently no way to know exactly how much the game has made selling on third-party sites or in bundles. We can only confirm how many copies have been sold directly on Steam.

After all, nothing can be as accurate as the numbers provided by the developers themselves. You should always sanity check all estimates before making any decisions

Some things to watch out for:

• Smaller games have less accurate estimates than the larger ones due to smaller sample size
• Free-to-play games generally have less accurate estimations
• Revenue estimates do not take into account revenue from any external sources and may not properly estimate revenue from steam bundles
• There may be bugs in our scraping or estimation algorithm, causing the estimates to be wildly off. Always sanity check any estimates

Conclusion

Overall, using all of these methods together is far more accurate than using review-based estimations only. It is important to note, however, that when conducting market research, these estimates should only be used as a supplementary information and should not be followed blindly. Obviously, there is still room for improvement here, and I will continue to work on the algorithm to improve it even more.

If you found this article helpful, check out our tool at gamalytic.com