Bug Bounty: the ultimate guide to a successful program

Thinking of getting into bug hunting, but unsure of where to start? Hackers, scope, triage… Here’s a step-by-step bug bounty guide to a successful program.

Bug bounty is still a young practice, so some wonder if it is an effective approach to security testing. The answer is simple: yes, but not without conditions.

When it comes to bug bounty, two questions stand between success and failure:

• Why?
• And how?

This article will not explore the why of bug bounty as a security test, nor its strengths and weaknesses. Nevertheless, it is imperative to understand the exercise before jumping in, and to ensure that it is the most appropriate test for your perimeters and security needs.

If you’re still thinking about it, we’d recommend you take a look at some prior reading on the topic:

• Why and how to get started?
• The pros and cons of bug bounty for cybersecurity
• Penetration Test vs Bug Bounty: which approach is right for you?

If your choice of bug bounty is well thought-out: congratulations, you’ve come to the right place! But it’s not enough to create a bug bounty program: you have to do it properly. And you’ll see that there’s a lot of work to be done!

How to build your bug bounty program?

There is more to bug bounty than just running it, you need to know how. How to set up a successful bug bounty program? In the end, it comes down to three things:

1. The attractiveness of the bug bounty program;
2. The program management by the internal teams;
3. The development teams’ approach to remediation.

The first two points determine the quality of your relationship with the hackers, and the performance of the program. The third point determines the good reception of the bug bounty internally, and the transformation of detections into actionable results. Put together, they entirely condition the success or failure of your bug bounty.

These three points are the core of this article. Hang on, we’re gonna cover it all in detail.

I. The attractiveness of the bug bounty program

To be effective, a bug bounty program must be attractive. In other words, it must make ethical hackers want to participate; to invest time in trying to find vulnerabilities in your perimeters rather than in the neighbor’s.

No hackers participating = no bugs = ineffective bounty. It’s as simple as that. What does it mean in practice?

A bug bounty program is attractive if:

1. the scope is clear and exhaustive;
2. the communication with the hunters is fast and transparent;
3. the triage of the vulnerability reports is quick;
4. payments are sent promptly after validation of a report;
5. Incentives are competitive;
6. the program’s visibility is aligned with the assets’ maturity.

1. The scope

The scope of a bug bounty must be clear and exhaustive. These are the rules of your program. They explain what the hunters can and cannot do, which vulnerabilities are valid and which are not – DDoS attacks for example.

The scope also specifies which products and applications are covered by the bug bounty – website, apps, etc. It goes without saying that the larger the scope, the more attractive it is for hunters. After all, the more mines to explore, the more likely they are to find the right vein.

The program manifesto should be as detailed as possible, even though it may take some time to write. It is the future basis of your entire bug bounty, your showcase to the hunters. A poorly written or incomplete program can cause more management time than it would have taken to write.

2. Communication with the hunters

Communication with the hunters must be quick and clear.

Clear communication is the basis of a good relationship between human beings. Answering to hackers allows them to be oriented in their research, which is as much in your interest as in theirs. Also, don’t forget that hunters come from all over the world. We don’t all have the same background, we don’t all speak the same language – even if English remains the reference language in exchanges. The diversity of profiles is what makes the strength of bug bounty. Clear communication helps to clear up ambiguities or misunderstandings.

A quick communication shows your interest in bug bounty and your seriousness in the exercise. Asking a hacker to participate in your program means asking them to take some of their time to find vulnerabilities in your perimeters. It’s asking them to take on your challenge rather than someone else’s. Would you accept the challenge of someone who doesn’t bother to answer you?

3. Triage of reports

The speed at which reports are sorted is crucial to the success of a bug bounty program.

Imagine that a hunter takes the time to submit a vulnerability, and he/she doesn’t hear from you for 2 months, 6 months or more… There is little chance that he will submit a second report. Worse, other researchers may decide not to participate in your program. Again, the speed with which you triage the reports sent to you is a reflection of how serious you are about bug bounty.

If you reject a report, take the time to explain why. No need to write novels, but an explanation is always welcome. Remember that a hunter who sends you a report is someone who has taken the time to investigate your environments. Take the time to respond. Also, explaining the rejection of a vulnerability often allows hunters to direct their future research – which can only be beneficial to you.

4. Speed of payment

All work deserves to be paid, it’s as simple as that. Once a report submitted by a hacker is accepted, the bonus must be paid quickly. One bug = one bounty, that’s the very foundation of bug bounty.

Among the best security researchers, some have made bug bounty their profession, their livelihood. And like everyone else, they have their share of expenses to pay at the end of the month. The speed of payment is therefore an argument that can make all the difference between your program and the neighbor’s.

5. The competitiveness of incentives

In other words, the value of the rewards for a vulnerability. The more competitive they are, the more the best players will participate in your bug bounty. The amount of the reward depends directly on the criticality of the identified vulnerability – low, medium, high, critical. The amount associated with each criticality should be clearly displayed in the program guidelines.

It is best to start with modest amounts and then increase as the program grows. This will keep hunters interested in your program, as well as scaling the bounties with difficulty. The older a program is, the more “easy” vulnerabilities will already have been found. It is therefore logical that the bounties increase with the difficulty.

There is no need to worry too much about the amount of the rewards. Unless you go for a self-hosted bug bounty, you will be counseled by the bug bounty platform you will have chosen – Yogosha for instance! The platform will advise you on the rewards according to the ones offered by other programs, the number of participants, the size of your company, its security level, the sector of activity, etc.

Furthermore, don’t think that the amount of the incentives is the most important criterion in the attractiveness of a bug bounty. Too many organizations are discouraged by the lack of a substantial budget. But not everyone is Google or Facebook, and hunters know this very well. Many members of the Yogosha Strike Force will tell you that communication and speed of triage are more important than the bonus pool.

6. The visibility of the bug bounty

The visibility of a bug bounty determines how many hackers can potentially participate. A program can be public or private.

• A public bug bounty is open to everyone on the web, anyone can participate.
• A private bug bounty is only open to hackers who are specifically invited to participate.

Public bug bounty and private bug bounty

With a public bug bounty, the number of participating hackers is potentially unlimited. But this is at the expense of the confidentiality of the program – some sensitive organizations prefer to maintain a form of secrecy, such as financial institutions. Moreover, a public bug bounty does not allow to screen the hunters according to their talent. There will be experienced hackers, but also beginners. To put it simply, choosing a public bug bounty means choosing quantity over quality.

On the other hand, a private bug bounty allows organizations to:

• decide on the number of participating hackers;
• of their level of expertise;
• control the volume of reports, and therefore the workload of the triage team. Obviously, a public bug bounty open to the whole world will generate more reports than a private bug bounty with 200 invited hackers. However, the reports will not necessarily be of better quality – the number of invalid reports, false positives and duplicates can be colossal with a public program. This is called the SNR, the Signal-to-Noise Ratio.

About the SNR of public bug bounties: The lower the SNR, the poorer the overall quality of the reports sent. As an example, 95% of the vulnerabilities reported to Facebook and Google at their public programs’ launch were invalid. As for Uber’s public bug bounty, it had a SNR of 1:6 in its first days, with 20% of duplicates and only 161 reports accepted for 2,030 received.

Bug bounty: public vs private platforms

Most bug bounty platforms are public. They sometimes offer the possibility to conduct invitation-only programs, but their community of hackers is by nature public. In other words, any hacker can create an account on these platforms, regardless of his or her skills, expertise or even identity.

In contrast, Yogosha is a private platform. This means two things:

• Our community of hunters, the Yogosha Strike Force, is entirely private. Every member of the YSF has proven his/her worth by passing our entrance tests – which are demanding to say the least, since only 20% of the participants join us. (See How to join Yogosha?)
• All bug bounty programs conducted on our platform are confidential. Regarding visibility, our customers have two choices:
  1. A program can be open to all members of the Yogosha Strike Force.
  2. Or it can be reserved to a group of hunters specifically selected, depending on their skills for example, which may correspond to the technical requirements of some environments. Some hunters are skilled in cryptos, while others lean towards IoT or the cloud.

“We have a very pragmatic vision of security, and we found the same approach at Yogosha. For the tests we had to do, we were put in touch with profiles that had the necessary skills on this technology to be relevant.” – Christophe Cuyala, Teréga’s Operational Security Manager

The visibility of bug bounty to control the competition between hunters

Behind the question of the bug bounty’s visibility is actually another one: the competition between hunters. In other words, how to multiply the number of participants to increase the number of detections, without demotivating the hackers to participate out of fear of an excessively harsh competition?

Before participating in your bug bounty, a hacker will always estimate his chances to find a vulnerability in your perimeters. Would you be more likely to participate in a lottery with 100 participants, or 10,000 participants? It’s exactly the same with bug bounty, except that talent is more important than luck.

Too much competition leads to a lower ROI for the hackers. Joining an overcrowded bug bounty increases the probability of sending a duplicate report – which has already been submitted by someone else – and therefore not getting the reward.

But then, how to control the competition between hunters? It’s simple, it comes back to the visibility of the bug bounty. It is the visibility that will determine the balance between the real number of participants and the potential number of vulnerabilities to find.

Match the program’s visibility to the maturity of the targeted assets

A public bug bounty will inevitably attract more participants, and therefore be more competitive than a private bug bounty. The fewer hackers invited to a program > the more likely they are to find a vulnerability individually > and the more likely they are to participate.

However, inviting not enough researchers would decrease the searching capacity, and thus the detections. The visibility of the bug bounty must therefore be consistent with the maturity of the targeted assets.

As a general rule, the best way to proceed is as follows:

• Start with a private bug bounty, inviting only a handful of hunters. They will clear the most obvious vulnerabilities in the youngest scopes, while allowing time for internal teams to get familiar with the bug bounty exercise.
• Invite more hunters to participate over time and CI/CD. Bringing in new blood, with fresh eyes, increases the likelihood of identifying critical, complex and time-consuming to find vulnerabilities. Increasing the bounties over time is also a good thing.
• If necessary, consider a public bug bounty, usually after several years. The time spent in small groups should have identified most of the vulnerabilities, and strengthened the overall security of the assets. If the detections become too scarce with a private bug bounty, it can be interesting to open it to the whole community of researchers.

Competition, cooperation and chainbug

Let’s stress that competition between hackers is not intrinsically a bad thing, as long as it is well balanced. Competition is part of the game.

A competitive program will push some hunters to collaborate, to work in groups to find complex vulnerabilities. On the other hand, a less competitive program will encourage researchers to chainbug: to try to string together vulnerabilities that are seemingly minor to create a more critical result.

• If the competition is fierce, hackers will tend to submit a bug as quickly as possible in fear that someone else will do it first – the famous duplicate report;
• If the competition is less, hunters can take the time to experiment, to chain together different vulnerabilities to increase the final criticality, and maximize the bounty.

II. The program management by the internal teams

You now know everything that makes a good bug bounty on paper. But only a solid in-house management of the program will make the difference between theory and practice.

Inform internal teams of the launch of a bug bounty

It is imperative to notify the internal teams of the launch of a bug bounty. One might want to test the responsiveness of the teams by playing the surprise card, but this is not the role of a bug bounty. There are other forms of security testing to gauge the company’s resilience – Red Team exercises, for example.

Starting a bug bounty without alerting internal teams could set a weak foundation for the future. 

• The detections can be significant, and the remediations numerous. It is useless to create an unexpected workload and stress for your staff, who could be caught off guard.
• The bug bounty exercise could be badly perceived later on. Some people may even never have heard of it. Introducing your teams to it with a flood of hackers coming out of nowhere might not be the best idea.
• The primary role of bug bounty is to detect vulnerabilities. Mixing exercises and objectives sends the wrong signal.

Evangelizing other departments

It’s not enough to inform tech-savvy teams, such as the IT and sec teams. For a bug bounty to be successful within the company, it is important to communicate with all the business units that might be involved.

• The legal department will probably have some questions about ethical hacking. Take the time to answer them, to address sensitive issues;
• The marketing team may want to communicate about the bug bounty launch. It may take some time to prepare the communication elements;
• The HR department may want to use the bug bounty to reinforce the employer brand for tech profiles.

Don’t forget that not all these people have a tech background. A developer will surely understand very quickly what a bug bounty is all about, but it may take a bit longer for the company’s lawyers. It is up to you to take the time to explain what bug bounty is and why you are implementing it. Otherwise, the practice could be misunderstood at best, and poorly received at worst.

When communicating with non-technical teams, don’t hesitate to use the term “security researcher” rather than “hacker” or “ethical hacker”. Attitudes are changing, but the word “hacker” still has a negative connotation in the collective mind. Some people might be afraid or reluctant to accept the bug bounty, whereas “security researcher” can reassure and bring a certain credibility to a newcomer audience.

Training internal teams in bug bounty

It may seem obvious, but it’s better to be safe than sorry. It’s not enough to notify the tech teams of the launch of a bug bounty, you have to explain what it is. Not everyone is familiar with the subject.

Explain the benefits of bug bounty, the role of hunters in the continuous detection of vulnerabilities. It is essential that teams understand the concepts of bug bounty, as well as its lexicon. Triage, vulnerability reports, bounties… So many terms that it can be useful to define before everyone joins the hunt.

To train teams in bug bounty:

• share resources that you have found useful – this article for example?
• organize a workshop on the subject – before and after the bug bounty is launched;
• once the first few weeks are over, don’t hesitate to share some numbers internally, to demonstrate the program’s results;
• if you think big, why not organize a Live Hacking Event? A live bug bounty competition, during which your teams and hackers can meet for a few days.

Designate one or more Program Owners

For your bug bounty to be successful, it is essential to appoint one or more program owners to manage the project and communicate with ethical hackers.

You can have a permanent program owner, or a rotational role within the team. This will help to:

• share the extra workload of bug bounty – communication and triage;
• initiate everyone to the exercise;
• get everyone on board, so that successes and failures are those of a team and not of one person;
• a passive training of each individual, who can increase his/her skills through contact with hackers.

On this matter, let’s mention that the Yogosha Vulnerability Operations Center (VOC) allows to restrict access and roles, within teams but also within the different entities that can compose a same organization. This silo-based structure allows for careful regulation of everyone’s involvement and visibility into the company’s assets security.

Read also: What is a Vulnerability Operations Center (VOC)?

Triage: managed internally or entrusted to a bug bounty platform?

Sorting out vulnerability reports sent by hunters is the most time-consuming task of a bug bounty. There are two approaches to triage.

1. Internalize the triage

Some organizations decide to assign the triage to their internal teams. Most often a member of the security team, or a developer with a keen interest in cybersecurity. Internalizing triage has many advantages:

• You keep control over your bug bounty;
• You ensure the confidentiality of vulnerability reports;
• You have a complete overview on feedbacks, both positive and negative;
• You ensure the relevance of the triage. After all, your teams know your products better than any outsider, and are therefore in the best position to judge the real business impact of a vulnerability.

This is testified by Antonin Garcia, Veepee‘s CISO, who entrusted the triage to the OffSec team: “Since they do pentesting, they have the necessary finesse to gauge vulnerabilities. They know how to qualify criticality, and engage in a dialogue if there is disagreement with a hunter.“

If you trust your internal teams to do the triage, stress the importance of communication with the hunters. It is absolutely crucial to maintain a healthy and transparent exchange with hunters, whether reports are accepted or rejected. This clear communication is the foundation of your entire relationship with the community.

2. Entrust the triage to a bug bounty platform

The second solution is to entrust the triage of the reports to the platform you have chosen to host your bug bounty. This solution has the double advantage:

• to relieve internal teams of this workload, which can be quite tedious depending on the flow of reports;
• to allow all organizations to do bug bounty, including those that do not have the in-house expertise or resources to triage vulnerability reports.

If you outsource report triage to a platform, find out about the triage teams. Most bug bounty platforms have in-house triage teams. They are therefore judge and jury in case of disagreement or conflict with a hacker.

At Yogosha, we guarantee our impartiality in triage by working with the most prestigious consulting firms. Our Managed Services extend well beyond triage, with comprehensive and scalable solutions for big players. We design end-to-end security strategies, from build to remediation.

Update the program regularly

It is important to regularly update your bug bounty manifest – especially the scope – for two reasons:

1. Making several updates a year rather than just one – or worse, at launch and then never again! – reflects your program’s activity. Hunters will be more likely to participate in your bug bounty if they see that you are serious about it, that your program is alive and evolving.
2. Updating your bug bounty with new releases allows you to direct the hunters’ searches, to focus their efforts on what you want to test. A new feature has gone into production? Update your scope, and tell them!

III. The development team’s approach to remediation

This is the end of the tunnel, the last sprint. The previous steps condition the performance of your program; but this is where you transform the detections into actionable results. The role of bug bounty is simple: to identify vulnerabilities – especially the most critical ones. But what is the point of detection if nothing is fixed?

Remediations made by developers are crucial, as they are the ones that will ultimately reinforce the overall security of assets. It is therefore important to ensure that these teams consider the bug bounty feedback in the best possible way.

Prioritize remediations, build processes

It is imperative to establish clear and precise processes, in cybersecurity perhaps even more than elsewhere. They will allow your teams to consider remediations in the best possible way, without having to make decisions in a hurry.

Remediation processes must be written and communicated to the teams prior to the bug bounty. They must enable them to answer at least the following questions when a vulnerability is accepted:

• Who? What are the human resources to be mobilized? Are the teams and collaborators in charge of remediation different depending on the products, environments and technologies?
• When? How long does it take to fix? It is imperative to set remediation timeframes according to the criticality of vulnerabilities. A critical vulnerability must be addressed as soon as possible, whereas a low one can probably wait.
• How? How to patch the vulnerability? What are the technical requirements and skills needed?

These points are the bare essentials to put in place. The more detailed your processes are, the better. It may be useful to align with the company’s incident response plan, to define on-call times in case of a critical vulnerability detection, or to agree on the feedback to be given after remediation.

Free up time for teams to remediate

This may seem obvious, but it’s important to free up time for development teams to remediate. This is a task on its own, not something to add to the usual workload or to do “when you have a moment”.

Beware of the launch rush. There are often many detections during the first few weeks of a bug bounty, and the first few hours can be particularly intense. It is important that teams are prepared to respond, at least psychologically. Don’t launch a bug bounty thinking that it will not generate any feedback.

Furthermore, the detection pace of the bug bounty must be in line with the remediation capacity of the organization’s teams. If high and critical vulnerabilities are identified faster than the team can fix them, stress is inevitable – and burn-out if it persists.

This brings us back to the importance of processes for prioritizing remediation actions. Furthermore, with Yogosha, it is possible to pause programs to regulate the flow of reports. Be sure to notify the participating hunters, and not to pause the bug bounty in total opacity. Some hunters may have been about to send a report, and may not understand why the program is frozen. Again, communication with hunters is essential.

Bug bounty is a continuous assessment, not a punishment

Bug bounty is a continuous monitoring of the organization’s digital assets, not a punishment for its development teams. Vulnerabilities detected by bug bounty should not be seen as an indication of failure, or presented as evidence of a poorly done job. This will only make your internal teams angry at the bug bounty and the hunters. 

There will always be vulnerabilities in any digital environment. Sometimes by design, sometimes not. Asking a developer to never create any flaws is like asking a human being to never make mistakes. This is simply impossible. On the other hand, it is possible to advocate improvement.

Bug bounty to detect the weaknesses of a team

Bug bounty is an excellent tool to understand the strengths and weaknesses of a development team. A course can be beneficial if you notice that the same type of vulnerabilities are regularly exploited by hunters, or that a particular pattern emerges. Similarly, a product’s notable resilience to a common class of vulnerabilities can be a reflection of a team’s expertise, which could be shared with other collaborators.

Bug bounty to raise awareness of cybersecurity among developers

Thanks to the regular contact with ethical hackers, bug bounty is an excellent field training in cybersecurity for your development teams. Bug bounty offers a new perspective on their work, not with the eye of a developer but with the one of an attacker. Developers will be better able to integrate good security by design by learning from bug bounty, by discovering the real attack vectors used by hunters.

Bug bounty is a fun and concrete approach to get devs interested in digital security. This is what Julien Reitzel, the Lead Security Offensive at Veepee, explains:

“I didn’t expect it, but we’ve noticed that vulnerabilities reported through bug bounty get more interest from teams. […] When we say that a vulnerability comes from bug bounty, they think, “ah, but that means anyone on the Internet can do that to my application! And on top of that, we had to pay someone for finding that particular vuln.“

By following these guidelines, you should be ready to tackle bug hunting in the best possible way. If you decide to pursue the bug bounty adventure, feel free to contact us!