Bug bounty: benefits and drawbacks for cybersecurity

A bug bounty program can do wonders for an organization’s cybersecurity, but it’s not a magic bullet. Let’s take a look at its strengths and weaknesses.

Bug bounty is an approach to security testing that has won its spurs, along with pentesting and automatic scanners. But it’s still a relatively young practice – a decade or so old, take it or leave it – and still little-known to many actors in the cybersecurity scene. 

Indeed, there is a certain opacity surrounding it, often fostered by the sugar-coated discourse of the bug bounty platforms themselves. As a result, while some doubt its effectiveness, others see it as the ultimate security test, the one that could supersede all others! Yet, as with most things, the truth lies in moderation.

Let’s face it: bug bounty is not a magic bullet. It’s a method of detecting vulnerabilities which, like all others, has its pros and cons. If you rush headlong into bug bounty, you’re heading straight for the wall. It’s crucial to understand its strengths, objectives and mechanisms, as well as its limitations and shortcomings. Only then can bug bounty be exploited to its full potential, and enhance the security of systems and organizations.

Read also: Bug Bounty, why and how to get started?

These few lines aim to explain the strengths and weaknesses of bug bounty, as realistically as possible, without glitter nor dust.

The benefits of bug bounty

1. The power of numbers

Calling on the community of researchers to test a product’s security enables to expand viewpoints and expertise. It’s obvious that, given the same amount of time, several hundred people will identify more vulnerabilities than a small security team.

It’s a fact that bug bounty has a striking force that’s hard to match. However, beware of the “armies of ethical hackers” narrative promoted by some bug bounty platforms. Yes, bug bounty makes it possible to mobilize hundreds of researchers to test the security of an asset. But no, it doesn’t provide access to an endless pool of thousands of hackers who are all experts in cybersecurity. If you’d like to dig deeper into the subject, we suggest you read the following article.

2. Diversity of skills

Hackers have a wide range of profiles, and each researcher comes with their own skills and professional experience. Where one hacker may be more comfortable working on APIs, others may be more inclined to test web apps or cloud configurations. Similarly, some hackers will be more skilled at detecting this or that type of vulnerability. This diversity of talents means that strategies and angles of attack can be more numerous, and assets can be tested more exhaustively.

3. A continuous, flexible approach to security testing

A bug bounty is usually an ongoing program, providing a permanent watch on the security of an organization’s assets. Vulnerabilities are reported throughout the year, and the program remains relevant for the entire lifecycle of the targeted software. Bug bounty thus complements DevSecOps approaches, with a security testing layer that never sleeps.

Moreover, this continuity gives bug bounty a flexibility of execution that other forms of testing, such as pentesting, do not possess. For example, researchers can work on a new release or feature as soon as it is available – all it takes is an update to the program to alert them and guide their research.

4. In-depth vulnerability detection

As bug bounty is not time-bound, it can uncover critical vulnerabilities in depth. These vulnerabilities are difficult to identify, as they require time and a good knowledge of the environments involved. It goes without saying that a hunter who can work over the long term will be able to carry out more sophisticated attacks than a pentester with only two weeks and a methodology to follow.

Bug bounty gives testers the time they need to familiarize themselves with environments and set up more advanced attack scenarios, exploiting multiple vulnerabilities to maximize security impact. This is known as chainbug, and researchers themselves have an interest in it, since the reward will be higher. Some hackers even choose to work as a team for the most elaborate attacks (and to share the bounty), thus engaging in a form of red teaming.

“We’ve done a lot of pentests. But with bug bounty, it’s different. You’re not waiting for a report, you’re waiting for a critical vulnerability. And with Yogosha, criticals have almost always been relevant.” – Antonin Garcia, CISO at Veepee

5. A cost-effective solution for organizations

Bug bounty is a test with a pay-per-result logic. Hackers are not paid for their research time, but rewarded for the fruit of their research. With bug bounty, organizations pay for exploitable vulnerabilities, those that have a real impact on security. Expenditure is therefore directly proportional to the number and, above all, the severity of the detections. And in the absence of valid vulnerabilities, organizations don’t have to spend a penny.

This is what differentiates bug bounty from other forms of testing, such as penetration tests, which are invoiced on a per-unit basis, and for which the billing is identical regardless of the number of vulnerabilities identified.

6. Measurable ROI

For many CISOs and security managers, convincing top management to allocate budgets to cybersecurity is a recurring challenge. All too often, cybersecurity is still seen as an expense rather than a vital investment. Here, bug bounty has the advantage of being a security test with a measurable ROI.

Detections are assessed according to their criticality, typology, number and financial value, which depends directly on their impact had they been exploited by a malicious actor. In addition, the entire bug bounty business model is advantageous to organizations, since there is no expenditure in the absence of results.

This ability to quantify effectiveness and results is a real plus for CISOs wishing to highlight the value of cybersecurity to decision-makers. An effective bug bounty should enable any security manager to say, “This quarter, we spent this much to identify X vulnerabilities, which has allowed us to avoid X potential attacks that could have cost the company this much.“

7. Passive security training for developers

Bug bounty has other benefits as well, though these are difficult to quantify. One of these is raising awareness and training development teams in cybersecurity issues.

Awareness-raising, because bug bounty makes vulnerabilities real for development teams. Putting their code into production inevitably leads to researchers identifying weaknesses. So there’s an incentive to produce secure code, since there’s the assurance that multiple researchers will be there to assess its robustness.

Passive training, as developers can increase their skills on security topics through contact with researchers. Bug bounty allows direct exchange with the bug hunters, who can advise on corrective measures to be taken.

“Yogosha’s researchers have a certain ethic, and they share their experience with development teams. With bug bounty, we improve the know-how and sensitivity of in-house developers. It’s a continuous training effect.” – Philippe Puyou Lascassies, CISO at Teréga

The drawbacks of bug bounty

Now that we’ve seen the strengths of bug bounty, let’s move on to its weaknesses. As you might expect, bug bounty is not a flawless magic bullet.

1. A method not suited to young perimeters

Let’s be clear. Bug bounty is not appropriate for assets that are not yet sufficiently mature in terms of digital security.

It is imperative to verify assets using other forms of testing, such as pentesting or automatic scanners, before submitting them to a bug bounty. These prior tests help to eliminate the “bulk” of vulnerabilities, which are the most common and easiest to detect.

Launching a bug bounty on a scope that’s too fresh means taking the risk of receiving far too many vulnerability reports. And this usually comes with three drawbacks: 

• time-consuming triage and analysis of reports by security teams – unless this task has been outsourced to the bug bounty platform;
• too much remediation work for development teams, whose motivation and performance may be affected;
• a hefty bill for the organization, since each valid report leads to a reward, whereas most obvious vulnerabilities could have been cleared for a fixed cost with a pentest.

On the other hand, bug bounty is ideal for testing mature environments, where there are complex vulnerabilities that take a long time to discover.

Read also: Penetration Test vs Bug Bounty, which approach is right for you?

2. A demanding practice for organizations

Bug bounty is a highly effective method of detecting vulnerabilities, but it’s also very demanding for organizations.

Let’s put it bluntly: with a pentest, all you have to do is pay a service provider and let them do their job. With bug bounty, companies have to invest time, and human, technical and financial resources. To be effective, bug bounty requires a good understanding of the discipline AND a good knowledge of one’s own security environment.

It’s essential to ask yourself whether bug bounty is the most relevant test for your environments (see previous point), and also whether you have the resources to implement it properly. Here are a few questions to ask yourself before jumping in:

1. Do my development teams have the bandwidth they need to remediate any vulnerabilities discovered?
2. What budget can I allocate to bug bounty rewards, and is it sufficient to motivate researchers to participate in my program? Is it competitive with other programs in my industry?
3. Who will be responsible for triaging and assessing the vulnerability reports received?
4. Who will be in charge of communication with the researchers?

The last two points can be solved with a managed bug bounty program. In this case, the bug bounty platform itself takes care of communicating with its hackers and screening reports.

3. A test that introduces a degree of randomness

This is not really a downside of bug bounty – in fact, it’s more of a benefit – but it can be disconcerting if you’re not prepared for it.

Getting involved in bug bounty means accepting a part of uncertainty. And this is precisely the strength of bug bounty: it introduces a degree of randomness that is lacking in more methodological approaches such as penetration testing. With bug bounty, you know what you’re looking for (critical, high-value vulnerabilities), but you don’t know what you’re going to find, or how.

A well-constructed bug bounty helps to frame the exercise and guide detections. The program tells the hackers what they can and cannot do, but it’s important to give them some freedom. Bug bounty is a test that needs to be fairly flexible and tolerate a reasonable dose of hazard. Only then can it be truly effective.

The pros and cons of bug bounty in a nutshell

Bug bounty is an approach to security testing that offers a number of advantages, including:

• the strength in numbers provided by the community of ethical hackers;
• the diversity of their skills;
• a continuous, flexible approach to asset security;
• in-depth detection of critical vulnerabilities;
• a pay-for-results approach that benefits organizations;
• a measurable ROI that helps promote security to decision-makers;
• awareness-raising and training of development teams in cybersecurity issues.

However, bug bounty is not a miracle recipe, to be prescribed unconditionally for all digital assets. It is an approach to security testing that needs to be thought through in advance, and integrated into an overall security strategy. Bug bounty should complement but not replace other forms of testing, such as pentests and vulnerability scans. 

A bug bounty program must be well thought-out and well-constructed. It must be tailored to the company’s security objectives, organizational specifics, resources and experience. A poorly designed program can quickly become a burden rather than an asset. It’s a good idea to seek expert advice, and it’s essential to choose the right bug bounty platform.

Bug bounty is a security test that can do wonders, provided it’s carried out properly. There are many considerations to take into account, and it’s not advisable to rush headlong into it.

That’s why, for those who want to take the plunge, we’ve put together a comprehensive guide to bug bounty for a successful program.