Learn how to choose the right bug bounty platform for your organization by exploring the differences between private and public platforms.
Bug bounty is a well-known method of detecting vulnerabilities. But while the practice itself is well established, the way in which bug bounty platforms operate can be somewhat opaque to the uninitiated.
There are two types of bug bounty platform:
- Public platforms – the majority, since this is the historical model.
- Private platforms, which can be counted on the fingers of one hand.
We won’t give you an exhaustive list of all the different players, but you will find here a list of most of the bug bounty platforms.
Understanding the differences to choose the right bug bounty platform
Public and private platforms come with their own way of working, their own pros and cons. Their mechanics is a complex topic, in which it’s easy to get lost, somewhere between thoughts on crowdsourced security and the marketing rhetoric of each player.
The purpose of this article is to help companies and institutions make an informed choice when selecting a bug bounty platform.
We’re not impartial on the subject, so we obviously can’t tell you which is the best platform. What we can do, however, is give you the keys to understanding the inner workings of public and private platforms, and their philosophy, so that you can forge your own opinion.
It’s important to note that we’re not going to focus here on the nature of bug bounty, but rather on the bug bounty platforms themselves. So, if you’re not familiar with bug bounty as a security test, we advise you to read the following article before going any further.
The misleading narrative of the “armies of ethical hackers”
Public bug bounty platforms are built on the same narrative: an entire army of ethical hackers exists in the shadows, ready to help any organization. Just look at the messages they broadcast (yes, they are real ones): “our skilled hacker community is over one million strong“,” “access a virtually unlimited pool of ethical hackers“, “it takes a crowd to defeat a crowd” – we actually like this last one tho, it’s clever!
These messages are based on a fair observation: there is a global shortage of talent in cybersecurity. No one can argue against that, it’s a reality – see this article by Forbes. But in the face of this problem, public platforms put forward a solution that can’t possibly exist. If there’s a talent shortage, then where do these armies of experts come from? You’d agree there’s food for thought.
Yes, bug bounty is about strength in numbers, and mobilizing several researchers to test the same perimeter. But there’s a gap between saying that bug bounty makes it possible to mobilize a substantial number of hackers, and saying that bug bounty gives access to an endless pool of specialists.
The real challenge is access to skilled researchers, not their numbers
Where all platforms agree is that it is essential to address the lack of qualified cybersecurity professionals. But this narrative of armies of hackers, whether true or not, is not relevant to the real need of businesses, which is access to quality rather than quantity.
The real challenge in bridging the skills gap is access to the right experts at the right time, not access to an inexhaustible pool of researchers with disparate know-how. What companies need is to be put in touch with qualified researchers, not thousands of wannabes.
Two different models for recruiting researchers
This difference in vision of how to help organizations overcome the skills gap lies at the very heart of the dichotomy between public and private platforms. Indeed, the main difference between the two types of platform is the massive size gap between their communities of researchers.
Public platforms promise communities of 40,000 to 1 million researchers, while private platforms announce no more than 2,000 experts. The gap is considerable, and the choice seems easy. But it’s important to understand that these figures don’t hide the same reality.
Don’t put words in our mouths: the figures put forward by public platforms to talk about their communities are real – or at least we don’t deny that they are. But to understand why the public platforms’ communities are so large, we first need to understand their hacker recruitment model.
Public platforms: open registration for all
Registration on public bug bounty platforms is unrestricted – yes, hence the name “public platforms”.
In other words, anyone can create a researcher’s account – whether beginner or experienced – and access a number of bug bounty programs. We encourage you to try it for yourself. Admission to these platforms is subject to no skills test and, for some, no identity check.
Private platforms: selection of researchers based on tests
On the other hand, private platforms are selective. In other words, you can’t take part in their bug bounty programs until you’ve passed a certain number of technical tests and identity checks.
For example, you can create an account on the Yogosha platform, and you’ll find that you can’t access any programs. To do so, you’ll first need to prove your identity and apply to take the entry tests to join the Yogosha Strike Force – and succeed, of course.
Public platforms: the number of registrants does not reflect their expertise
The lack of selection at the door of public platforms leads to one thing: the difficulty of proving the expertise of researchers. When a public platform says it has 50,000 hackers, it’s really saying that 50,000 accounts have been created on its platform. If anyone can register as a researcher, you’ll agree that numbers can’t be a reflection of any quality – good or bad.
Yes, there are some excellent researchers registered on public platforms, some of whom are among the world’s elite. But there’s no guarantee that these are the experts who will be working on your perimeters. They represent only part of the overall public platforms community, which is also made up of beginners, students, mid-level researchers, script kiddies and inactive accounts. In short, it’s a mixed bag: some of it very good, some of it very bad.
Inexperienced researchers: a problem for organizations
For an organization, the participation of the most junior researchers can have a number of consequences, more or less important.
The first is, of course, the irrelevance of vulnerability reports. Some companies – such as large corporations, banks and governments – have achieved such a high level of security that it is obvious that novice researchers will be of no help to them. The real strength of bug bounty is the ability of the sharpest minds to identify critical and in-depth vulnerabilities, and not to overwhelm companies with detections akin to those of a properly calibrated scanner.
Detections by novice researchers can :
- At best raise obvious issues – “have you heard of Log4j?“, poorly configured HSTS, secure cookies, you name it.
- At worst, they can generate enough noise to monopolize in-house security teams.
Indeed, too many vulnerability reports leads to tedious triage work. And if the reports turn out to be irrelevant, it’s simply a waste of time for the teams in charge of screening them – often a member of the organization’s security team.
Another problem with rookie hackers is that they can disrupt live assets (website, mobile app, etc.) with unreasonable, low-quality attacks, which can lead to degraded response times or even bring down the entire system without being able to stop it or find out who is causing the error. Which brings us straight to the next point: identifying researchers, both legally and technically.
The identity of security researchers
In addition to the hackers’ expertise, there’s the matter of their identity. Verifying the identity of ethical hackers raises issues of confidentiality, which may or may not be a problem depending on the sensitivity of organizations. Ethical hacking has flourished thanks to a degree of anonymity for researchers, which has often been beneficial for their protection. It’s a broad topic, which deserves its own treatment, and the point here is not to stigmatize the anonymity of hackers.
Nevertheless, not all companies are comfortable with the idea of complete strangers testing the defenses of their systems. In addition to confidentiality issues, the identity of researchers raises legal questions in many countries, including France.
Here, it’s up to you to check how each platform addresses the issue. All private bug bounty platforms verify the identity of their researchers, but not all public platforms do the same – or do so under certain conditions, for example when inviting participants to a sensitive program.
In France, identity verification through a KYC (Know Your Customer) procedure and the use of an escrow account are mandatory. For the record, the identity of every Yogosha Strike Force researcher – French or otherwise – has been verified by a KYC procedure carried out by an independent third party.
From a technical point of view, the inability to isolate researcher activity can also raise issues for IT teams, especially for production environments. After all, how can you distinguish the activity of an ethical hacker from that of a true attacker? If possible, the best thing to do is to provide researchers with a mirror copy of the asset, without any sensitive data. Let’s mention here that our VPN also enables SOC teams to easily identify all traffic originating from our hunters.
Researcher community activity rates
All bug bounty platforms disclose the number of registered researchers, but very few of them disclose the activity rate of their community.
Is a hacker who signed up 7 years ago still active? Maybe he is, or maybe he’s found a job, is using another platform, or has simply retired from the world of ethical hacking.
Is a novice researcher who signed up for training still around after several months? Perhaps she has become a brilliant expert, and an active and recognized member. Or maybe she’s given up her offensive security career, because it’s a difficult and demanding apprenticeship that not everyone can see through to the end.
The activity rate of platforms is a key issue, as it determines the activity of their bug bounty programs. Without active researchers, there’s no activity, and no vulnerabilities uncovered – and this is true for all platforms.
E-Book: Bug bounty, the ultimate guide to a successful program
Learn how to build your Bug Bounty program, make it attractive and leverage hackers to identify high-risk vulnerabilities.
Two questions to ask when choosing a bug bounty platform
If you’re an organization considering bug bounty, there are two questions you should ask a platform that promises you several thousand researchers:
- How can you guarantee the expertise of the researchers who will be working on my perimeters?
- How many researchers have been active in your community in recent months?
If you struggle to get convincing answers, you’ll know where to stand.
For our part, we can answer you right here, since it’s the combination of activity and expertise that makes our community so effective.
1. Yogosha guarantees the expertise of its researchers through a series of practical and pedagogical tests, which assess both their technical skills and their ability to communicate clearly and effectively with our customers. Our entry tests are demanding, and on average only 20% of candidates join the Yogosha Strike Force each month. (See How to join Yogosha?)
2. We race for activity, not “who has the biggest community”. To date, the Yogosha Strike Force hovers between 800 and 1000 active members every month – yes, bug hunters have summer vacations too –, and we’re proud of it. This selective, restricted and active community is a direct reflection of our pledge to organizations: the solution to the skills gap isn’t access to thousands of researchers, but connecting you to the right experts at the right time.
The trick of private or invitation-only bug bounty programs
Faced with the question of how they select researchers, the traditional response of public platforms is private bug bounty programs, sometimes referred to as ” invitation-only” programs. It’s this dual use of the word “private” to refer to programs AND platforms that can lead to confusion.
A private (or invitation-only) bug bounty program is one that is not open to the entire community of a platform, but only to researchers expressly invited to participate. Today, all platforms, both public and private, offer private bug bounty. However, this terminology means different things depending on the platform using it.
Private bug bounty on a public platform
A private bug bounty program run on a public platform allows the selection of hackers from a community that is itself public and open to all.
In other words, for public platforms, invitation-only programs are a first level of hacker screening based on various criteria ; such as notoriety and past performance on the platform, or affinity with the technologies involved.
Private bug bounty on a private platform
A private bug bounty program run on a private platform enables the selection of researchers from a selective community that has already carried out an initial evaluation of their expertise.
For private platforms, invitation-only programs therefore represent a second layer of screening, since the technical skills of the researchers have already been assessed. Invitation-only programs allow other factors to be taken into account – adjusting the number of participants, choosing the researchers with the best knowledge of a given technology according to the specificities of the program, etc.
All platforms have a role to play in crowdsourced security
By now, you’re probably thinking that this article is an indictment of public platforms. Well, we’ve got something good to say too.
Public platforms, a gateway to ethical hacking
The point here is not to denigrate the hackers registered on public platforms, but to highlight certain points of communication of these platforms to talk about their community. We’ll say it loud and clear: there are some excellent researchers on public platforms, but not all registered researchers are excellent. The nuance is important.
Some hackers are true OffSec geniuses, others are just average. Still others are just starting out in cybersecurity, as students or self-taught. The ethical hacking community needs this diversity. Without this diversity of researchers, ethical hacking is doomed to disappear. And that’s where public platforms have an essential role to play – a role that private platforms like us will never be able to fulfill.
A role in training tomorrow’s experts
Public platforms have a training role to play in the sphere of ethical hacking. Aspiring hackers need a place to train, improve and perfect their approaches and techniques. Ethical hacking needs platforms that are open to everyone, so that tomorrow’s future talents can emerge.
Yes, there are also ethical hacking training platforms like HackTheBox and the PortSwigger Academy. These are invaluable to the hacking ecosystem, and provide an opportunity to experiment with a wide variety of exercises. But public bug bounty platforms are also an important learning vector, since :
- they are open to all researchers, beginners and experienced alike;
- they are free to use for all researchers;
- they provide public bug bounty programs that can be used for training in real-life environments.
Without public platforms to support the training of researchers, the talent shortage would inevitably become more acute – and private platforms would have fewer experts to recruit. But without private platforms, some organizations with particularly demanding security needs would not be able to experiment with bug bounty in its most rigorous form.
Ultimately, bug bounty depends on a fragile ecosystem – that of ethical hacking – whose future and vitality depend on the platforms and researchers themselves, but also on the position of organizations and governments towards this practice. This isn’t the subject of these few lines, but we could talk about the importance of ethical hackers as whistle-blowers, and the duty of governments to guarantee them legal protection that lives up to the name.
So, public or private platforms, which is the better model?
There are two approaches to bug bounty, both of which address the same problem: the need for organizations to find cybersecurity professionals.
- Public bug bounty platforms offer a quantitative response, with open communities of several thousand researchers with varying degrees of expertise.
- Private platforms, such as Yogosha, have chosen to respond with quality. Their communities may be much smaller, but all their members have been selected on the basis of technical and pedagogical tests, in addition to strict identity checks.
So, yes, we stand by our guns because we’re convinced that our approach is the right one. We’ll say it again, but the real challenge in overcoming the skills gap is to connect with the right researchers at the right time, not to access huge pools of hackers with disparate skills.
But the truth here remains the same: public and private platforms offer two approaches to bug bounty, which are two different solutions to the same challenge: the cybersecurity skills gap.
At the end of the day, the real question for an organization is not to know which platform model is best, but rather to know which one best suits it. This is an open question for all security leaders considering bug bounty. Which approach, private or public, do you most identify with? Which is more in tune with your security needs?
For those who feel more at home with the private, selective model, we have just one last thing to say.
Yogosha is the only entirely private platform to offer bug bounty in Europe, period.