How Much Are GitHub Stars Worth to You?

The best and most obvious way to judge an open-source project is to look at the code but this can be kind of tedious and sometimes you don’t like what you see there, so an alternative that we have all naturally developed on our own or have been advised to, is to see how many people have starred a project, and then pick the one with the most stars.

“For example, React.js has 207K stars compared to Angular’s measly 88K stars, so we can conclude that React.js is a better framework” — Ben Awad

While the above is a joke (not that React isn’t a better frontend framework), it is not a good idea to judge the quality of an open-source project by how many stars it has.

How many times have you gazed upon an open source project, and wondered fabulously about the correlation between the number of stars garnered and the actual value of the project?

If it’s zero, then ok, you’re a better person than me.

But if not, then allow me to introduce you to my recent experiments and research. It’s likely that this will be your first encounter with this bizarre market.

Buying GitHub Stars

Yep! That’s a thing. Just as purchasing Instagram likes is a known used marketing technique, buying GitHub stars is equally tangible.

But I won’t just tell you that buying GitHub stars is real. That would be so disappointing, right?

So I did it! I have tried two platforms. I won’t say any names or links, as I would never advocate for such a thing.

I thought it would be a fascinating and very educational experiment (not for my wallet, though!). So let’s dive into it!

Premium Stars

Bear with me. I know that you’re thinking: “what the hell?! There are different star types??!” Yes, there are!

Premium Stars are the kind that is much more expensive, the most legit-looking, and most notably, those stars have a higher chance of staying over the long term.

So I put on Incognito mode and started hunting using my second identity “Boogeyman”.

And so it costed me 20 Euros for 25 stars which is €0.8/star, it’s expensive (not if you deduct it from your taxes), but I’m gonna show you how they look like in a minute.

It took six hours for my order to complete, and the accounts look legit; each has a profile picture, different companies that they work for, a couple of repositories, and a contribution to one or more open-source projects, next to being a GitHub member for over a year.

Cheap Stars

“Whenever you see premium goods, remember, the bargain bin isn’t far behind!” — Probably me

There are also cheaper options like this one:

This package costed me 8.19 Euros for 100 stars which is €0.08/star.

As you can see from the receipt, my order is number #57189, so it’s definitely not something that’s only used every once in a while 🙂.

Now, what is the difference?

Well, the biggest one is that those are brand new accounts — they were created at the time of my order. They don’t have any fake personal information or repositories or contributions.

And after a month, they are all gone. GitHub detected and banned them.

What’s crazy is that you can reach out to them , and they are going to redo your order for free 🙂

Detecting Fake Stars

Then I wondered if any tools can expose those repositories 🤔

And I discovered Astronomer, a tool to detect illegitimate stars from bot accounts on GitHub projects.

It works by fetching data from every GitHub user who starred a repository and computing how likely it is that those users are real humans. It works using their “Trust algorithm”.

The trust score is computed based on different factors:

• The average amount of lifetime contributions among stargazers
• The average amount of private contributions
• The average amount of public-created issues
• The average amount of publicly authored commits
• The average amount of publicly opened pull requests
• The average amount of public code reviews
• The average weighted contribution score (weighted by making older contributions more trustworthy)
• Every 5th percentile, from 5 to 95, of the weighted contribution score
• The average account age — older ones are more trustworthy

So, I’ve tried Astronomer on three repositories to see the results.

Premium Stars:

Fetching contributions for 69 users up to year 2013
Building trust report...ok

Averages                             Score           Trust
--------                             -----           -----
Weighted contributions:              18268             B
Private contributions:               370               A
Created issues:                      91                A
Commits authored:                    259               C
Repositories:                        41                A
Pull requests:                       36                A
Code reviews:                        6                 C
Account age (days):                  1619              B
5th percentile:                      12                A
10th percentile:                     16                C
15th percentile:                     20                D
20th percentile:                     20                E
25th percentile:                     24                E
30th percentile:                     24                E
35th percentile:                     24                E
40th percentile:                     28                E
45th percentile:                     332               D
50th percentile:                     1106              B
55th percentile:                     1692              B
60th percentile:                     2770              A
65th percentile:                     6422              A
70th percentile:                     10958             A
75th percentile:                     15598             A
80th percentile:                     26596             A
85th percentile:                     34116             A
90th percentile:                     69967             A
95th percentile:                     85147             A
----------------------------------------------------------
Overall trust:                                         B

Cheap Stars:

Fetching contributions for 24 users up to year 2013
Building trust report...ok

Averages                             Score           Trust
--------                             -----           -----
Weighted contributions:              856             E
Private contributions:               21              E
Created issues:                      3               E
Commits authored:                    87              E
Repositories:                        4               E
Pull requests:                       3               E
Code reviews:                        1               E
Account age (days):                  36              E
5th percentile:                      3               D
10th percentile:                     3               E
15th percentile:                     3               E
20th percentile:                     3               E
25th percentile:                     3               E
30th percentile:                     3               E
35th percentile:                     3               E
40th percentile:                     3               E
45th percentile:                     3               E
50th percentile:                     3               E
55th percentile:                     3               E
60th percentile:                     3               E
65th percentile:                     3               E
70th percentile:                     3               E
75th percentile:                     3               E
80th percentile:                     4               E
85th percentile:                     4               E
90th percentile:                     4               E
95th percentile:                     4               E
----------------------------------------------------------
Overall trust:                                       E

Legit Stars (as a control check):

Fetching contributions for 139 users up to year 2013
Building trust report...ok

Averages                             Score           Trust
--------                             -----           -----
Weighted contributions:              118821            A
Private contributions:               2407              A
Created issues:                      72                A
Commits authored:                    1300              A
Repositories:                        74                A
Pull requests:                       101               A
Code reviews:                        69                A
Account age (days):                  2538              A
5th percentile:                      145               A
10th percentile:                     711               A
15th percentile:                     2726              A
20th percentile:                     4144              A
25th percentile:                     8535              A
30th percentile:                     13650             A
35th percentile:                     20728             A
40th percentile:                     23322             A
45th percentile:                     28244             A
50th percentile:                     37636             A
55th percentile:                     42536             A
60th percentile:                     55776             A
65th percentile:                     78956             A
70th percentile:                     93676             A
75th percentile:                     129164            A
80th percentile:                     194463            A
85th percentile:                     231380            A
90th percentile:                     286798            A
95th percentile:                     559934            A
----------------------------------------------------------
Overall trust:                                         A

So, that’s scary. The premium stars scored decently as B, but the cheap ones resulted in the lowest score — E. Who knows, maybe there’s an ultra-premium stars service that’s scoring an A?

Lessons Learned

As you shouldn’t judge a book by its cover, you shouldn’t judge an open-source project by the number of stargazers. As we saw in this article, this is an option that they are not legitimate. It’s like any other social media platform where fake accounts and fake likes exist.

Instead, ask your colleagues or your community on Twitter why you should pick this project over another. You can also start a new discussion or create an issue on GitHub asking for other people’s experiences. And if that’s not enough — you can review the project’s code. Though, we all know, that’s not something we have time and energy for on a daily basis.

Choosing a high-quality open-source project can be a challenging task. However, there are key factors to consider when making such decisions. As The Guild whole structure was created to ship long term open source projects, we’ve also have developed a sense on how to identify good open source projects, based on long-term sustainability and collaborative ideas from everyone involved.

We recommend considering the following:

1. Long-term sustainability: Look for projects that demonstrate a commitment to continuous development and maintenance. A good place to start is the contributors tab on GitHub, as it shows you the whole timeline span of a project. For example GraphQL Mesh and GraphQL Codegen are consistantly being worked on for years, Hive is increasing work gradually and consistanly, on GraphQL Yoga you can see how it started, then abandoned, and later returned strongly after The Guild took over maintaince from previous maintainers and the Nextra project significantly increased work after we’ve joined as maintainers for v2.
2. Community engagement: Assess the level of community involvement and collaboration within the project. Here you can check out the discussions tab on GitHub or the GitHub community graph.
3. Code quality: Evaluate the quality of the codebase, including readability, maintainability, and adherence to best practices.
4. Responsiveness: Check how responsive the project maintainers are to issues, pull requests, and community feedback. The pulse tab on GitHub can help with that.
5. Popularity and reputation: Consider the project’s reputation within the community, including the number of contributors, stars, and positive reviews.

By following these guidelines, you can increase your chances of selecting a high-quality open-source project. At The Guild, we strive to embody these principles in our own work, creating sustainable and impactful solutions for the long term.

And, there you have it!

Got some seed money for your next startup, and you wonder how to spend them? Throw it into building truly worthwhile projects. That’s an investment you won’t regret. 😉