Bug Bounty: the differences between public and private platforms

Learn how to choose the right bug bounty platform for your organization by exploring the differences between private and public platforms.

Bug bounty is a well-known method of detecting vulnerabilities. But while the practice itself is well established, the way in which bug bounty platforms operate can be somewhat opaque to the uninitiated.

There are two types of bug bounty platform: 

• Public platforms – the majority, since this is the historical model.
• Private platforms, which can be counted on the fingers of one hand.

We won’t give you an exhaustive list of all the different players, but you will find here a list of most of the bug bounty platforms.

Understanding the differences to choose the right bug bounty platform

Public and private platforms come with their own way of working, their own pros and cons. Their mechanics is a complex topic, in which it’s easy to get lost, somewhere between thoughts on crowdsourced security and the marketing rhetoric of each player.

The purpose of this article is to help companies and institutions make an informed choice when selecting a bug bounty platform.

We’re not impartial on the subject, so we obviously can’t tell you which is the best platform. What we can do, however, is give you the keys to understanding the inner workings of public and private platforms, and their philosophy, so that you can forge your own opinion.

It’s important to note that we’re not going to focus here on the nature of bug bounty, but rather on the bug bounty platforms themselves. So, if you’re not familiar with bug bounty as a security test, we advise you to read the following article before going any further.

The misleading narrative of the “armies of ethical hackers”

Public bug bounty platforms are built on the same narrative: an entire army of ethical hackers exists in the shadows, ready to help any organization. Just look at the messages they broadcast (yes, they are real ones): “our skilled hacker community is over one million strong“,” “access a virtually unlimited pool of ethical hackers“, “it takes a crowd to defeat a crowd” – we actually like this last one tho, it’s clever!

These messages are based on a fair observation: there is a global shortage of talent in cybersecurity. No one can argue against that, it’s a reality – see this article by Forbes. But in the face of this problem, public platforms put forward a solution that can’t possibly exist. If there’s a talent shortage, then where do these armies of experts come from? You’d agree there’s food for thought.

Yes, bug bounty is about strength in numbers, and mobilizing several researchers to test the same perimeter. But there’s a gap between saying that bug bounty makes it possible to mobilize a substantial number of hackers, and saying that bug bounty gives access to an endless pool of specialists.

The real challenge is access to skilled researchers, not their numbers

Where all platforms agree is that it is essential to address the lack of qualified cybersecurity professionals. But this narrative of armies of hackers, whether true or not, is not relevant to the real need of businesses, which is access to quality rather than quantity.

The real challenge in bridging the skills gap is access to the right experts at the right time, not access to an inexhaustible pool of researchers with disparate know-how. What companies need is to be put in touch with qualified researchers, not thousands of wannabes.

“Through Yogosha, we managed to find talented people.”
– Eric Vautier, CISO at Groupe ADP (Paris Airports)

Two different models for recruiting researchers

This difference in vision of how to help organizations overcome the skills gap lies at the very heart of the dichotomy between public and private platforms. Indeed, the main difference between the two types of platform is the massive size gap between their communities of researchers.

Public platforms promise communities of 40,000 to 1 million researchers, while private platforms announce no more than 2,000 experts. The gap is considerable, and the choice seems easy. But it’s important to understand that these figures don’t hide the same reality.

Don’t put words in our mouths: the figures put forward by public platforms to talk about their communities are real – or at least we don’t deny that they are. But to understand why the public platforms’ communities are so large, we first need to understand their hacker recruitment model.

Public platforms: open registration for all

Registration on public bug bounty platforms is unrestricted – yes, hence the name “public platforms”.

In other words, anyone can create a researcher’s account – whether beginner or experienced – and access a number of bug bounty programs. We encourage you to try it for yourself. Admission to these platforms is subject to no skills test and, for some, no identity check.

Private platforms: selection of researchers based on tests

On the other hand, private platforms are selective. In other words, you can’t take part in their bug bounty programs until you’ve passed a certain number of technical tests and identity checks.

For example, you can create an account on the Yogosha platform, and you’ll find that you can’t access any programs. To do so, you’ll first need to prove your identity and apply to take the entry tests to join the Yogosha Strike Force – and succeed, of course.

Public platforms: the number of registrants does not reflect their expertise

The lack of selection at the door of public platforms leads to one thing: the difficulty of proving the expertise of researchers. When a public platform says it has 50,000 hackers, it’s really saying that 50,000 accounts have been created on its platform. If anyone can register as a researcher, you’ll agree that numbers can’t be a reflection of any quality – good or bad.

Yes, there are some excellent researchers registered on public platforms, some of whom are among the world’s elite. But there’s no guarantee that these are the experts who will be working on your perimeters. They represent only part of the overall public platforms community, which is also made up of beginners, students, mid-level researchers, script kiddies and inactive accounts. In short, it’s a mixed bag: some of it very good, some of it very bad.

Inexperienced researchers: a problem for organizations

For an organization, the participation of the most junior researchers can have a number of consequences, more or less important.

The first is, of course, the irrelevance of vulnerability reports. Some companies – such as large corporations, banks and governments – have achieved such a high level of security that it is obvious that novice researchers will be of no help to them. The real strength of bug bounty is the ability of the sharpest minds to identify critical and in-depth vulnerabilities, and not to overwhelm companies with detections akin to those of a properly calibrated scanner.

Detections by novice researchers can :

• At best raise obvious issues – “have you heard of Log4j?“, poorly configured HSTS, secure cookies, you name it.
• At worst, they can generate enough noise to monopolize in-house security teams. 

Indeed, too many vulnerability reports leads to tedious triage work. And if the reports turn out to be irrelevant, it’s simply a waste of time for the teams in charge of screening them – often a member of the organization’s security team.

Another problem with rookie hackers is that they can disrupt live assets (website, mobile app, etc.) with unreasonable, low-quality attacks, which can lead to degraded response times or even bring down the entire system without being able to stop it or find out who is causing the error. Which brings us straight to the next point: identifying researchers, both legally and technically.

The identity of security researchers

In addition to the hackers’ expertise, there’s the matter of their identity. Verifying the identity of ethical hackers raises issues of confidentiality, which may or may not be a problem depending on the sensitivity of organizations. Ethical hacking has flourished thanks to a degree of anonymity for researchers, which has often been beneficial for their protection. It’s a broad topic, which deserves its own treatment, and the point here is not to stigmatize the anonymity of hackers.

Nevertheless, not all companies are comfortable with the idea of complete strangers testing the defenses of their systems. In addition to confidentiality issues, the identity of researchers raises legal questions in many countries, including France.

Here, it’s up to you to check how each platform addresses the issue. All private bug bounty platforms verify the identity of their researchers, but not all public platforms do the same – or do so under certain conditions, for example when inviting participants to a sensitive program.

In France, identity verification through a KYC (Know Your Customer) procedure and the use of an escrow account are mandatory. For the record, the identity of every Yogosha Strike Force researcher – French or otherwise – has been verified by a KYC procedure carried out by an independent third party.

“The ID check of the hunters by Yogosha has been really interesting. We contact each other, we can have discussions.”
– Antonin Garcia, CISO at Veepee

From a technical point of view, the inability to isolate researcher activity can also raise issues for IT teams, especially for production environments. After all, how can you distinguish the activity of an ethical hacker from that of a true attacker? If possible, the best thing to do is to provide researchers with a mirror copy of the asset, without any sensitive data. Let’s mention here that our VPN also enables SOC teams to easily identify all traffic originating from our hunters.

Researcher community activity rates

All bug bounty platforms disclose the number of registered researchers, but very few of them disclose the activity rate of their community.

Is a hacker who signed up 7 years ago still active? Maybe he is, or maybe he’s found a job, is using another platform, or has simply retired from the world of ethical hacking.

Is a novice researcher who signed up for training still around after several months? Perhaps she has become a brilliant expert, and an active and recognized member. Or maybe she’s given up her offensive security career, because it’s a difficult and demanding apprenticeship that not everyone can see through to the end.

The activity rate of platforms is a key issue, as it determines the activity of their bug bounty programs. Without active researchers, there’s no activity, and no vulnerabilities uncovered – and this is true for all platforms.

So, public or private platforms, which is the better model?

There are two approaches to bug bounty, both of which address the same problem: the need for organizations to find cybersecurity professionals.

• Public bug bounty platforms offer a quantitative response, with open communities of several thousand researchers with varying degrees of expertise.
• Private platforms, such as Yogosha, have chosen to respond with quality. Their communities may be much smaller, but all their members have been selected on the basis of technical and pedagogical tests, in addition to strict identity checks.

So, yes, we stand by our guns because we’re convinced that our approach is the right one. We’ll say it again, but the real challenge in overcoming the skills gap is to connect with the right researchers at the right time, not to access huge pools of hackers with disparate skills.

But the truth here remains the same: public and private platforms offer two approaches to bug bounty, which are two different solutions to the same challenge: the cybersecurity skills gap.

At the end of the day, the real question for an organization is not to know which platform model is best, but rather to know which one best suits it. This is an open question for all security leaders considering bug bounty. Which approach, private or public, do you most identify with? Which is more in tune with your security needs?

For those who feel more at home with the private, selective model, we have just one last thing to say.

Yogosha is the only entirely private platform to offer bug bounty in Europe, period.