Table of Contents
Threat-Led Penetration Testing (TLPT) costs DORA-regulated entities time and money. Let’s take a look at how best to prepare for a successful outcome.
Nota Bene: This article is the fourth and last chapter of our guide to security testing for DORA-regulated entities, which we recommend you read from the beginning if you haven’t already.
First of all, a quick reminder of what Threat-Led Penetration Tests (TLPT) are.
TLPTs are enhanced security tests, mandated by Article 26 of the Digital Operational Resilience Act (DORA), and intended for financial entities whose failure would have systemic effects. In practical terms, TLPTs are large-scale Red Team exercises, designed to test the entire attack surface of the targeted organization — i.e. its physical, human and digital dimensions.
It’s essential to understand the nature and requirements of TLPTs before you can grasp their implications for the security managers of regulated entities. If you’d like to delve deeper into the subject before reading on, we suggest you read the following article:
Read also: DORA: Everything About Threat-Led Penetration Testing (TLPT)
The Dual Purpose of TLPTs
Threat-Led Penetration Testing (TLPT) plays a key role in DORA’s mission to raise the overall level of digital resilience of financial entities in the European Union.
They have a dual function, which is to:
- enable designated financial entities to assess their level of security in real-life situations;
- enable the competent authorities — and through them the EU and Member States — to verify the actual resilience of entities that are strategic to the proper functioning of the financial system, by monitoring the effectiveness of their resilience testing program.
TLPTs are therefore the final exams of resilience testing, like a Bachelor’s degree at the end of one’s studies. But there’s a downside for regulated entities — and here, allow us to proceed with this dubious academic metaphor.
You can continue reading this chapter published as an article on our blog, or download the full DORA-guide to security testing below in PDF format for easier reading.
DORA: A Guide to Security Testing for Regulated Entities
A 60-page compliance guide to walk security managers of DORA-regulated entities through the regulation's security testing obligations: the resilience testing program, and Threat-Led Penetration Testing (TLPT).
Bridging the Gap Between Resilience Testing and TLPTs
Before facing their final exams, students can take preparatory tests, to practice and assess their ability to pass the real exam. Organizations regulated by DORA don’t have this luxury. Indeed, the Regulation requires:
- mandatory security measures and tests for financial entities — the resilience testing program introduced by Article 24;
- and control mechanisms for the authorities — Threat-Led Penetration Testing introduced by Article 26.
But DORA introduces no intermediate steps, no pre-TLPT training, no way for regulated entities to assess their maturity and ability to pass a Threat-Led Penetration Test.
Let’s be clear: DORA does not prohibit organizations from preparing themselves — quite the contrary — but it doesn’t offer a turnkey solution. So it’s up to each organization to set up its own preparatory test.
Some may question the usefulness of such an approach. After all, one could learn from a failed TLPT before organizing another, couldn’t one? Sadly, no.
DORA: A Complete Guide to Compliance for the Financial Sector
A 50-page guide to walk CISOs, DPOs and legal departments through the EU regulation. No mumbo jumbo, only useful and actionable insights.
A TLPT Costs Time and Money
Performing a Threat-Led Penetration Test costs time and money.
Time, because the processes associated with a TLPT are long and tedious. It requires evaluation and selection of Red Teaming and Threat Intelligence service providers, synchronization with the competent national authority — especially to approve the content of the exercise —, mobilizing the relevant internal teams (IT Department, Blue Team…), aggregating the results and reporting on them, and so on. Not to mention that the actual test itself is time-consuming, generally extending over a period of 6 to 12 months.
Money, because the various TLPT stakeholders need to be remunerated, starting with the Red Teaming and Threat Intelligence providers. Several indirect costs also need to be taken into account, such as the cost of mobilizing internal resources on this issue, including the CISO and operational teams, all the more so in the context of an in-house test — which can only be carried out under certain conditions; see our article on TLPT.
In short, at the risk of repeating: a TLPT is expensive, both in terms of time and money. It is therefore in the interest of financial entities to “succeed” in the exercise as much as possible, and not to organize several ones in a row; let’s point out here that the competent authorities can increase the frequency of TLPT for a given entity if they deem it necessary.
A Necessary Return on Investment for Regulated Entities
Implementing a TLPT therefore goes hand in hand with ROI considerations. A security test of this magnitude inevitably entails significant expenditure, which has an impact on the budget allocated to security. These costs also need to be approved by top management, as NIS2 and DORA call for accountability of the hierarchy in security matters, including through awareness of major cybersecurity projects. Clearly, a TLPT is not a routine exercise, and must be brought to the attention of top management.
In a way, investing such a budget reflects the professionalism of the CISO and his teams. They need to have sufficient confidence in the measures and tests previously implemented, at least enough to expect the TLPT’s conclusions to be positive.
Indeed, you don’t run a TLPT, with all its attendant costs, unless you’re reasonably confident that you’ll be able to handle it. As always when it comes to security: failure is a possibility, but thoroughness is not an option.
NIS2 Directive: Step-by-Step Guide to Compliance
A 40-page guide to walk CISOs, DPOs and legal departments through the directive. No mumbo jumbo, only useful and actionable insights.
The Importance of Pre-testing When Preparing for a TLPT
Given this ROI imperative, it’s in the interest of financial entities to assess their ability to withstand a TLPT (i.e., their resilience) prior to the test. This means finding an approach, a form of assessment that comes as close as possible to the modalities of a Threat-Led Penetration Test.
On top of this, there’s another requirement. Some organizations might be inclined to organize prior full-scale Red Team exercises, which would de facto be comparable to a TLPT. This is an effective solution, but it will involve more or less the same amount of time and money as the final test, and is therefore a luxury reserved for the best-endowed CISOs.
Ideally, a resilience test in preparation for a TLPT should therefore be:
- less costly and time-consuming than a TLPT, at the risk of duplicating efforts and costs;
- sufficiently close to the operational requirements of the TIBER-EU framework, in order to best assess the real capacity of financial entities to commit a TLPT budget with a high probability of a positive outcome.
Here, bug bounty seems to be an ideal solution, both from a security and a financial point of view.
TIBER-EU and Bug Bounty: A Shared Philosophy
A bug bounty is an offensive security test, which can be continuous or temporary. It’s a hunt for vulnerabilities, whereby an organization mobilizes the community of security researchers to test one or more digital assets — web and mobile applications, cloud, networks, IoT, etc. When a researcher identifies a vulnerability, the organization pays a reward according to the criticality. If no vulnerability is detected, the organization spends nothing.
Bug bounty shares the same philosophy as TIBER-EU testing: to help organizations understand and improve their resilience by simulating real-life attacks over an extended period, adopting the tactics, techniques and procedures used by malicious actors.
By way of illustration, here is an extract from the TIBER-EU framework as drafted by the European Central Bank (ECB):
“An intelligence-led red team test mimics the TTPs of real attackers on the basis of bespoke threat intelligence. In doing so, it looks to target the people, processes and technologies underpinning the critical functions of an entity in order to test its protection, detection and response capabilities with no foreknowledge. It allows the entity to understand its real-world resilience by stressing all elements of its business against the TTPs of the threat actors that are specific to its organization.”
– TIBER-EU Framework, Service Procurement Guidelines, ECB
The bug bounty thus provides five elements that characterize a TLPT performed in compliance with the TIBER-EU framework:
- the mobilization of a large number of security experts;
- allowing a diversity of profiles and skills;
- essential for a wide range of tactics and techniques;
- with a black-box approach and no prior knowledge of the target;
- and over a sufficiently long period.
A Resilience Test That Verifies All Precedents
Let’s be clear: bug bounty doesn’t allow for testing the entire attack surface of an entity — physical, human and digital — as a TLPT would. After all, it doesn’t involve physical intrusion attempts or social engineering methods (although it might).
But bug bounty goes much further than any of the other tests listed in Article 25 of DORA, be they scans or penetration tests, for which the scope and duration are predefined, and the experts less numerous.
Bug bounty enables in-depth testing of one of the three attack surfaces of financial entities — namely the digital attack surface, at the very core of DORA and digital resilience — through controlled testing with a Black Box approach on live production assets. These are all mandatory features for a TLPT in line with TIBER-EU. This real-life testing validates the effectiveness of the tests and security measures previously adopted by the entities.
Bug bounty can therefore be considered as:
- a digital operational resilience test in its own right, as part of the testing obligations introduced by Article 25 of DORA;
- but also, and most importantly, as the final resilience test that verifies all the previous ones, and validates the entities’ ability to face a TLPT.
They do bug bounty and share their stories.
Bug Bounty as a Springboard to TIBER-EU Testing
Bug bounty provides an intermediate step between resilience testing and Threat-Led Penetration Testing. It offers financial entities a smoother progression curve in security learning, enabling them to assess the measures they are putting in place, their actual resilience and maturity, and their ability to withstand a large-scale Red Team exercise as part of TIBER-EU.
Bug bounty should be seen as a mid-term check between annual operational resilience tests (DORA) and TLPTs (DORA, TIBER). It is an intermediary step that contributes to strengthening the digital resilience of entities, while adding a real-world dimension that facilitates the adoption of DORA and the TIBER-EU framework by the financial sector.
It’s important to understand that:
- Threat-Led Penetration Testing (TLPT) is a control mechanism for the authorities to assess the effectiveness of the resilience testing program of critical entities in the European financial sector;
- Where bug bounty is a control mechanism for the financial entities themselves, to assess the effectiveness of their resilience testing program, their ability to withstand a TLPT and, above all, to commit a budget with a good chance of success.
A Cost-Effective Solution for Organizations
As already mentioned, implementing a TLPT isn’t cheap. If additional pre-testing is to be introduced, it’s important to limit the cost so as not to strain the budget. With this in mind, bug bounty has the advantage of being an attractive solution for financial entities, since it adopts a pay-for-results logic.
Indeed, the whole philosophy of bug bounty is that security researchers are not paid for their research time, but according to their detections and their criticality. If the identification of a vulnerability leads to a reward, this also means that the absence of detection entails no expense for organizations.
In principle, bug bounty comes at a stage in the security journey where financial entities are expected to be robust, namely at the end of the resilience testing program required by DORA. If the security work has been done properly, there’s no reason why the bug bounty program should be particularly costly. On the contrary, the number of detections should be low, confirming the financial entity’s ability to carry out a TLPT.
Selecting Security Researchers According to TIBER-EU Criteria
Bug bounty allows for the selection of security researchers according to the requirements of the TIBER-EU framework, at least in the context of a program run with Yogosha. This is a direct result of the quality of our security researchers, and the high standards of our selection processes — Yogosha is the first and only entirely private platform to offer bug bounty in Europe.
Read also: Bug Bounty, the differences between public and private platforms
The Yogosha Strike Force (YSF) is a private, selective community of over 800 international security researchers:
- Specialized in finding the most critical vulnerabilities by simulating sophisticated hacker attacks.
- Experts in multiple asset types — Web and Mobile Apps, IOT, Cloud, Networks, APIs, Infrastructure…
- Holders of recognized cybersecurity certifications — OSCP, OSEP, OSWE, OSEE, GXPN, GCPN, eWPTXv2, PNPT, CISSP…
We select only the most talented researchers: only 10% of applicants are accepted, after passing technical and redactional exams. Identity and background checks are also carried out.
As part of a bug bounty carried out prior to a TLPT, we introduce additional selection criteria for participating security researchers, directly modeled on the selection criteria for Red Team members as dictated by TIBER-EU.
As an example, here are some of these TIBER-EU criteria, and how we integrate them into our bug bounty programs for financial entities.
| Requirements for Red Team members as part of a Threat-Led Penetration Test (TLPT)(TIBER-EU Framework, ECB) | Selection criteria for Yogosha Strike Force (YSF) researchers eligible for financial bug bounty programs |
|---|---|
| “Sufficient experience of the red team members. Expectation for each member: at least two years of experience in red team testing” | We only select security researchers with a minimum of two years’ experience in Red Teaming, after examining their professional background. |
| “Up-to-date CV for each member of the team to be provided to the entity” | We can provide the CVs of the researchers selected to take part in the bug bounty. |
| “Multi-disciplinary composition of the red team, with a broad range of knowledge and skills,such as: business knowledge, red team testing, penetration testing, reconnaissance, threat intelligence, risk management, exploit development, physical penetration, social engineering, vulnerability analysis and combinations thereof” | The 800+ researchers of the Yogosha Strike Force (YSF) bring the diversity of knowledge and skills demanded here. We have proven, certified experts in each of these fields, from pentesting and threat intelligence to physical intrusion and social engineering. |
| “Background checks on each member of the red team are conducted by the RT provider” “Enhanced background checks are conducted as required by the national authorities” | All our researchers have undergone a KYC procedure conducted by an independent third party, and their identities and backgrounds have been duly verified. Additional checks can be carried out on a case-by-case basis for the most sensitive bug bounty programs. |
| “The red team members should have appropriate recognised qualifications and certifications – Certified Ethical Hacker (CEH), OSCP, OSWP, OSWE, CISSP, etc.” | We can select our researchers according to their qualifications and certifications – OSCP, OSEP, OSWE, OSEE, GXPN, GCPN, eWPTXv2, PNPT, CISSP… The size of the Yogosha Strike Force (800+ experts) enables us to source almost any profile. |
| Source: TIBER-EU Framework |
Yogosha as a Provider of Offensive Security Testing for DORA
Our community of certified security experts is not our only strength when it comes to bug bounty in the context of DORA, i.e. as an intermediate step between resilience testing and TLPT.
Firstly, we have expertise in DORA-related security testing. With our Pentest as a Service (PtaaS) solution, we are already conducting penetration tests as resilience testing for a number of players in the European financial sector.
We also offer a comprehensive Red Teaming service for Threat-Led Penetration Testing. We have also written a number of guides and articles — including the one you are reading right now — on topics relating to DORA and TIBER-EU.
We are therefore present from end to end in the DORA security testing chain, and have the expertise and knowledge to adjust each stage of testing (PtaaS, bug bounty and Red Teaming) according to compliance and security objectives, but also to the maturity of organizations and to their specific constraints — budget, scaling up, etc.
Secondly, we set ourselves the highest security standards. Yogosha is a French company, subject to the jurisdiction of the European Union. Our platform is hosted via 3DS Outscale and its SecNumCloud-certified sovereign cloud (the highest French security standard, established by the French CERT), and data is hosted on French soil.
Note that it is possible to use our SaaS platform as “single-tenant”, i.e. one instance of the application is entirely dedicated to you, in the region of your choice, allowing total compartmentalization of your data and customization options — logo, configurable URL, etc. — to suit your needs.
Another deployment option for the most sensitive financial entities: Yogosha is the only bug bounty provider in the world to offer a Self-Hosted platform. An ideal solution for organizations with the most stringent security requirements, who can host our platform wherever they like — private cloud, on premise — to retain full control over their data and execution context.
Thirdly, there’s our community of cybersecurity experts, the Yogosha Strike Force (YSF), but we’ve talked about that enough already.
By now, you’ve understood our intention: to tell you that we’re best placed to help you test your assets under DORA, whether:
- with our Pentest as a Service (PtaaS) solution as part of your resilience testing program [DORA, Article 25];
- with our bug bounty programs to assess the effectiveness of your previous resilience tests, the actual security level of your exposed assets, and your ability to deal with a TLPT;
- with our Red Teaming services as part of a Threat-Led Penetration Test (TLPT) [DORA, Article 26].
All that remains is to wish you good luck in securing your assets, and in organizing the various DORA-induced tests. And should you ever need advice at any stage, or even expert guidance, please feel free to contact us.
