DORA and the Digital Operational Resilience Testing Program

Find out all there is to know about the digital operational resilience testing program, a DORA-induced security testing obligation for all regulated entities.

Nota Bene: This article is the first chapter of our guide to security testing for DORA-regulated entities, which we recommend you read from the beginning if you haven’t already.

Digital operational resilience testing — hereafter referred to as resilience testing, for simplicity’s sake — is the first level of testing introduced by DORA. They are mandatory for all entities within the scope of the regulation, with one exception for microenterprises, to which we’ll return later.

You can continue reading this chapter published as an article on our blog, or download the full DORA-guide to security testing below in PDF format for easier reading.

A “Sound and Comprehensive” Program of Resilience Testing, With a Risk-Based Approach

Resilience tests are not isolated elements, but must form part of a “sound and comprehensive digital operation resilience testing program“, in the words of Article 24(3) of DORA. The creation of this program is the responsibility of each financial entity, as it must be adapted to the specifics of each one. So don’t expect a turnkey checklist that could be replicated for every entity in the EU.

The testing program must include “a range of assessments, tests, methodologies, practices and tools“, to which we’ll return later, and adopt a risk-based approach taking into account “the evolving landscape of ICT risk, any specific risks to which the financial entity concerned is or might be exposed, the criticality of information assets and of services provided, as well as any other factor the financial entity deems appropriate.” [DORA, A.24(3)]

Financial entities must also consider the principle of proportionality laid down in Article 4(2). The program must therefore be proportionate to their size and overall risk profile, as well as to the nature, scale and complexity of their services, activities and operations. In other words, DORA does not imply the same obligations for a major international bank as for a modest-sized insurance company. 

Let’s point out here that this philosophy sometimes makes large companies grumble, as it seems out of touch with the realities of the digital age and Big Data. Indeed, the volume of data held by a company isn’t necessarily proportional to its size — a phenomenon accentuated by certain regulations in the making, such as Financial Data Access (FIDA), which regulates open access to financial data.

Furthermore, more flexible obligations don’t mean a total absence of duties. Smaller companies are still subject to security testing obligations, and DORA’s focus on third-party security will mean that a modest company providing services to a larger institution will, as a knock-on effect, have to carry out extensive testing — particularly in terms of Threat-Led Penetration Testing (TLPT).

A Program to Be Included in the ICT Risk Management Framework…

The ICT risk management framework introduced by Article 6 is a central element of the legislation. It must detail all the elements put in place by the regulated entity to protect :

• all ICT and information assets, i.e. software, hardware or servers, but also data, which are assets in their own right.
• all physical components and infrastructures relevant to asset protection, such as premises and data centers.

Yes, it’s all software assets and data that are targeted here.

… And Into the Digital Operational Resilience Strategy

DORA is structured according to a fractal approach, and the section on testing is no exception. Indeed, the ICT risk management framework must itself include a digital operational resilience strategy, and it is this, among other things, that materializes the testing obligation.

“The ICT risk management framework shall include a digital operational resilience strategy setting out how the framework shall be implemented. To that end, the digital operational resilience strategy shall include methods to address ICT risk and attain specific ICT objectives, by […] implementing digital operational resilience testing.” – DORA, Article 6(8)

This may sound like an obvious statement, given that the risk management framework and the resilience strategy are so closely linked — the latter being induced by the former. Nothing new here for veteran CISOs, even if it’s always helpful to know where and how to organize your various policies and documentation. To find out more about the ICT risk management framework and the digital operational resilience strategy, we refer you to our DORA compliance guide.

More Flexible Requirements for Microenterprises

The testing requirements for microenterprises are not as stringent as for other financial entities. In fact, microenterprises benefit from a simplified ICT risk management framework [DORA, Article 16], and security tests are to be carried out rationally, according to their resources and risk exposure. The official wording of DORA is very clear on this subject:

“Microenterprises shall perform the tests referred to in paragraph 1 by combining a risk-based approach with a strategic planning of ICT testing, by duly considering the need to maintain a balanced approach between the scale of resources and the time to be allocated to the ICT testing provided for in this Article, on the one hand, and the urgency, type of risk, criticality of information assets and of services provided, as well as any other relevant factor, including the financial entity’s ability to take calculated risks, on the other hand.” – DORA, Article 25.3

Here again, we believe it’s important to temper this passage. Supply chain security is at the heart of major European cybersecurity legislation, including DORA, NIS2 and the Cyber Resilience Act (CRA). We can therefore expect small companies providing services to large financial institutions to be impacted by the requirements applicable to them in terms of third-party risk control, and therefore not to be exempt from any efforts.

Read also: NIS2 vs DORA: what differences and which legislation prevails?

Which Systems Should Be Tested?

The resilience testing program must be built in accordance with the risk management framework and its objectives. It’s not just a question of testing software assets, but ensuring that all assets are properly protected. This means testing the security of software, code, networks, cloud infrastructures, physical infrastructures and so on.

At Least Once a Year for Systems Supporting Critical or Important Functions

In the words of DORA, financial entities “shall ensure, at least yearly, that appropriate tests are conducted on all ICT systems and applications supporting critical or important functions.” [Article 24(6)]

Obviously, testing these systems once a year is the legal minimum, and more frequent testing is more than welcome. We find it odd that the regulator’s reasoning is based on the frequency of testing, whereas a continuous approach would be more in line with the practices and challenges of our time — agility, continuous deployment, interconnections, fast-growing functional complexity…

Furthermore, periodic testing offers only limited visibility, anchored in the moment, which leaves companies exposed to potential risks that may arise during the intervals between each test. Cyber attackers are adept at exploiting vulnerabilities swiftly, and relying on sporadic assessments simply isn’t sufficient in today’s threat landscape.

At the risk of repeating, it’s obvious that the annual tests required by DORA are the bare minimum. We’re talking here about testing systems that are essential to the proper functioning of financial entities that are themselves essential to our society; the security level must be exemplary.

By adopting a continuous testing approach, it’s not so much a question of being among the best students as of not joining the most lackluster, who will also be the most vulnerable. And when it comes to security, the consequences are far more dramatic than a bad grade.

If you’d like to delve deeper into this subject, we recommend you read our article Why It’s Time to Move on to Continuous Security Testing.

In-House or Outsourced Testing

Security tests can be carried out by independent service providers, or in-house, provided that no conflicts of interest arise during the test design and execution phases. DORA’s text is crystal-clear:

“Financial entities, other than microenterprises, shall ensure that tests are undertaken by independent parties, whether internal or external. Where tests are undertaken by an internal tester, financial entities shall dedicate sufficient resources and ensure that conflicts of interest are avoided throughout the design and execution phases of the test.” – DORA, Article 24(4)

Although DORA allows for internalized testing, the reality on the ground will probably lead most entities to juggle in-house skills with third-party service providers. Indeed, the tests to be run are numerous, eclectic and recurrent, and only large companies have the human, technical and financial resources required for total internalization.

The Types of Security Tests Advocated by DORA

Article 25 of DORA specifies some typologies of tests to run, such as:

• vulnerability assessments and scans;
• open source analyses – see our article Understanding OSINT, its tools, benefits and risks;
• network security assessments;
• gap analyses;
• physical security reviews;
• questionnaires and scanning software solutions;
• source code reviews where feasible;
• scenario-based tests;
• compatibility testing;
• performance testing;
• end-to-end testing;
• and penetration testing.

As you can see, the tests listed by DORA are many and varied. Some come under the heading of Threat Intelligence and aim to gather information on threats, while others intend to test the physical security of organizations, such as their offices or datacenters.

The same goes for tests designed to assess the cybersecurity of systems: they are eclectic, do not pursue the same objectives, and will not fit equally into an organization’s resilience strategy. For example, some are useful for assessing the security and/or proper functioning of applications produced (or acquired) by financial entities, such as code review and end-to-end testing, along with unit testing and integration testing. These are best performed upstream of production, during the development lifecycle (SDLC) as part of a DevSecOps approach.

Other tests check the robustness of the lines of defense, such as network security assessments, or even seek to detect vulnerabilities proactively, either automatically (vulnerability scans) or manually (penetration testing). These are part of what is known as Offensive Security, or OffSec.

The Role of Offensive Security in Test Strategy

Offensive Security is a proactive approach to cybersecurity. It complements defensive measures such as network protection and EDRs, but pursues different objectives. OffSec does not seek to protect digital assets, but to identify their vulnerabilities before malicious actors do. 

OffSec testing simulates attacks on assets by adopting the posture and methods of cyber attackers, in order to stress organizations’ defenses, identify potential weaknesses and assess their actual state of security. Offensive Security can be used to test almost any type of asset, from cloud infrastructures to web and mobile apps, APIs and IoT.

Read also: What is Offensive Security? An introductory guide to OffSec

It’s no coincidence that we’re telling you all this, but because we — Yogosha — specialize in Offensive Security Testing.

The Importance of Modernizing Security Testing

Pentesting is an approach recommended by Article 25 of DORA. It’s a technique that aims to simulate a cyber attack against an asset, in order to identify most of its vulnerabilities, and ensure its digital impermeability. Pentesting is a long-standing cybersecurity practice with which security teams are already familiar. But even tried-and-tested methods sometimes need updating.

The very principle of occasional penetration testing was introduced at a time when the content of applications and websites was stable, the resources to be audited were few, and the intrusion techniques to be explored were limited. Nowadays, a major banking application is updated every month, and intrusion techniques are legion.

Therefore, we need to imagine a collaborative system, supported by sufficiently large communities of cybersecurity researchers to ensure that no intrusion technique will be overlooked, to act continuously and to overcome the global shortage of skills.

Pentest as a Service (PtaaS)

With Pentest as a Service (PtaaS), Yogosha offers a different model by digitizing pentesting activities, allowing for real-time results and faster launches — a few days compared with 3 to 6 weeks for a traditional pentest. Our platform makes testing more efficient and smoother, whether it’s carried out in-house by your security teams, or by the Yogosha Strike Force, our community of 800+ certified security researchers.

Read also: Pentest as a Service vs traditional pentesting, which differences?

Pentest as a Service (PtaaS) addresses one of the major challenges posed by DORA and resilience testing: scaling cybersecurity. This is a key issue, which raises questions of process and cost, and which we’ll explore in more detail in the next chapter — link at the end of the page.

Bug Bounty

Penetration tests show their limitations once a sufficiently advanced level of security has been reached. They are time-bound, leave little room for original approaches, and depend on the expertise of a small number of people.

This is where Bug Bounty comes in, allowing organizations to experience real-world conditions, by mobilizing a community of security researchers to test all or part of their information system.

Read also: Pentest vs Bug Bounty, which approach is right for you?

Bug Bounty is a bug hunt based on a pay-per-result logic. Organizations pay monetary incentives — bounties — to researchers for each valid vulnerability they manage to find. The more critical the vulnerability, the higher the bounty. If no vulnerability is detected, the organization doesn’t have to spend a dime.

Researchers examine assets using a range of tools, tactics and procedures to detect in-depth, critical vulnerabilities that escape other forms of testing that are more calibrated, such as automated scanners. Discovered vulnerabilities are documented with their CVSS score, Proof of Concept (POC) and remediation advice, and reported live on a centralized vulnerability management platform, our Vulnerability Operations Center (VOC).

Bug Bounty is in line with the risk-based approaches recommended by DORA, NIS2, the CRA and other major cybersecurity legislation.

In the context of DORA, Bug Bounty has the dual advantage of being:

• a comprehensive resilience test to assess the security of exposed assets in controlled situations as close to the real thing as possible;
• a way of bridging the gap between resilience testing on the one hand, and Threat-Led Penetration Testing (TLPT) on the other. These two layers of testing introduced by DORA follow a very different curve of difficulty, and bug bounty acts as an intermediate step, enabling regulated entities to assess their ability to deal with a real attack, but also to undertake a Threat-Led Penetration Test — Cf. our article on the ROI of TLPTs.

Digital Operational Resilience Testing in a Nutshell

To sum up, resilience tests are:

• mandatory for all DORA-regulated entities;
• to be organized into a digital operational resilience testing program;
  • with a risk-based approach;
  • and built in line with the risk management framework and its objectives, i.e. to ensure that all the entity’s assets — both tangible and intangible, aka data — are properly protected, thereby calling for as many security tests as necessary;
• to be carried out at least once a year for all systems and applications supporting critical or important functions.

As mentioned earlier in this article, the regulation sets a minimum legal requirement to be met, but the very notion of one-off security tests is no longer attuned to modern development methods and security challenges.

To achieve state-of-the-art digital resilience, regulated entities should, wherever possible, implement continuous testing, while building on the solid foundations of proven methodologies. In this respect, Article 25 of DORA sets out a non-exhaustive list of tests to be carried out, including penetration tests.

Here, we’d like to remind you that Yogosha offers a Pentest as a Service (PtaaS) solution tailored to the needs of entities falling within the scope of DORA, in order to meet both the pentesting requirement and the need for continuous testing. Feel free to contact us for a quote or a demo.

Finally, it should be noted that the number, diversity and recurrence of security tests induced by DORA come with their fair share of difficulties. Indeed, such an increase in security testing is likely to result in a substantial rise in costs and workload for unprepared organizations.

This key issue is the subject of the second and next chapter of this guide to security testing: