Buyer’s guide: Breach and attack simulation (BAS) tools

Breach and attack simulation (BAS) defined

Breach and attack simulation (BAS) products help organizations understand their security posture by automating the tests of specific threat vectors. BAS typically uses MITRE ATT&CK and Cyber Killchain frameworks to emulate attacks such as network and infiltration, lateral movement, phishing, endpoint and gateway attacks, malware, and ransomware. The goal of these simulations is to test a company’s defenses against these attack vectors. BAS is complementary to but differs from red teaming and penetration testing.

You can think of BAS as going around to check whether all the locks on your doors work and whether your security cameras can detect people walking in front of them. Red teaming or penetration testing, by comparison, is like hiring someone to try to break into your house and steal your safe, who’d then go and find an unlocked window somewhere that you hadn’t even thought of.

[ Download our editors’ PDF breach and attack simulation tools buyer’s guide today! ]

In this buyer’s guide

• Breach and attack simulation (BAS) defined
• What the BAS market looks like
• Regulations drive the need for BAS
• What to look for in BAS products
• 9 leading BAS vendors
• What to ask before buying BAS
• Essential reading

As such, BAS is a good way to check that your security controls are working as intended, while penetration testing will find areas that those security controls might not cover that an attacker might use to get to their target. In addition, BAS simulations can use credentials and knowledge of internal systems to simulate not just external attacks, but insider threats.

BAS is also complementary to attack surface assessment (ASA). Whereas BAS focuses on ensuring that enterprise security controls — such as endpoint detection and response (EDR) — are working, ASA looks for all potential vulnerabilities and attack vectors. These technologies combine into a broader category that research firm Gartner calls exposure management, though different analyst groups may have slightly different definitions of these terms.

What the BAS market looks like

There are dedicated BAS providers, vendors that provide BAS as well as pen testing and other related services, and bigger cybersecurity vendors or service providers that have BAS as part of a broader cybersecurity portfolio, says Constellation Research analyst Chirag Mehta. And there’s likely to be more consolidation coming in this industry. “If you have a tool that can simulate attacks, the next logical step is to prevent those attacks,” Mehta tells CSO. But that requires integration between the different kinds of tools. “And it’s not that straightforward.”

By 2026, Gartner predicts more than 40% of organizations will rely on consolidated platforms or managed service providers to run cybersecurity validation assessments. But a possible downside of getting BAS from a vendor that offers XDR, for example, is that a company might not want to have the same vendor providing the defensive capability and the testing to see whether those defensive capabilities are, in fact, working as they’re supposed to.

Another big trend happening in this sector — as well as in all other areas of cybersecurity — is that vendors are looking for opportunities to integrate generative AI into their products. “When you look at what breach and attack simulation vendors are doing, there’s a lot of machine learning in there already,” says Forrester Research analyst Erik Nost.

Generative AI, he says, is the evolution. The first area where we’re likely to see generative AI is in the user interface. “The new applicability of generative AI is how cool it is to be able to interact with data,” Nost says. In the future, we might also see AI-powered capability to model threats based on intelligence, Nost says, or based on the types of attacks that users are most interested in, or that would be most likely to impact their companies. Generative AI could also be used to help companies understand the problems that BAS finds, to prioritize and to suggest specific remediation actions.

Regulations drive the need for BAS

According to Garner, typical BAS customers are financial institutions and insurance companies. But with increasing regulation, more and more companies are facing compliance requirements that focus on testing the efficacy of their cybersecurity controls.

BAS is typically a pricey product and not something bought by smaller companies with budget or operational constraints, adds Ilia Rabinovich, director of adversarial tactics at cybersecurity firm Sygnia.

What to look for in BAS products

The top features that companies should look for in BAS tools are the following:

• Representative attack vectors to simulate a wide range of attacks relevant to your company.
• Realistic attack scenarios that are similar to what attackers are actually using, using frameworks such as MITRE ATT&CK.
• Customizable scenarios to test unique aspects of your infrastructure.
• Automated testing so that the simulations can run regularly and efficiently without impacting operations or requiring additional headcount.
• Detailed reporting and analytics to help explain what the tests mean and identify areas that need improvements.
• Ability to scale to the current — and future — size and complexity of the enterprise environment.
• Ability to test across hybrid environments in production, which is critical for identifying how controls perform in real-world conditions.
• Ease of use and deployment, including out-of-the-box integrations with your existing security tools and platforms.
• Expert guidance and support, especially for companies that are new to BAS or that don’t have large, experienced security teams.
• And, of course, cost. BAS vendors typically don’t publish pricing information, and pricing models can vary. Make sure that the pricing structure is a good fit for your company’s use case.

9 leading BAS vendors

Enterprise technology research firm Expert Insights has curated a list of the top nine BAS vendors. The list takes into consideration key features such as threat emulation, reporting granularity, and ease of integration. Expert Insights’ top 9 are AttackIQ, Cymulate, Fortinet FortiTester, Mandiant Red Team Assessment, NetSPI Breach and Attack Simulation, Picus Security, RedScan Breach and Attack Simulation, ReliaQuest GreyMatter Verify, and SafeBreach Breach and Attack Simulation Platform.

Cymulate, Picus, AttackIQ, SafeBreach, Fortinet, and NetSPI are also among the top vendors, according to Gartner’s Peer Insights BAS tool rankings. The Gartner list is more comprehensive and has 17 vendors, but six of those have received no customer reviews while companies like XM Cyber and Keysight do not show up in Expert Insights’ list but have a high number in the Gartner ratings system.

AttackIQ: According to Expert Insights, AttackIQ’s core emulation platform replicates adversary tactics, techniques, and procedures in line with the MITRE ATT&CK framework. The company recently released the second generation of its managed breach and attack simulation-as-a-service platform, called Ready!, to make it easier and faster for companies to deploy a continuous security validation program.

AttackIQ’s other main BAS product is Flex, an on-demand, agentless testing service priced based on results rather than the number of tests or licenses.

AttackIQ is also known for its ability to test machine learning- and AI-based cybersecurity components. According to Carl Wright, AttackIQ’s chief commercial officer, the company is the only BAS vendor that offers both self-service and full-service BAS solutions and comanaged services. That means its products “can fit an organization’s needs, budget, and way they would like to consume the capability.”

AttackIQ is working on using AI to power its emulations. This will help customers accelerate their testing cycle, Wright says, “reducing their time to find and fix security gaps and autogenerate sigma rules for detection engineering use cases.”

Cymulate: According to Expert Insights, Cymulate is a leading continuous threat exposure management vendor. It is the top-rated vendor according to Gartner Peer Reviews. Like AttackIQ, Cymulate uses the MITRE ATT&CK framework and is known for its usability and user experience. Cymulate is deployed as a SaaS solution, with a private tenancy option for enterprises that require data segregation.

It takes three to four weeks to set up the integrations and deploy the tool, says Nir Loya Dahan, Cymulate‘s VP of product. The company plans to leverage generative AI soon to reduce this time to just minutes. It also plans to use generative AI to automatically create attack scenarios, to reduce thousands — or hundreds of thousands — of attack simulation results into a clear mitigation strategy, and to explain to security teams how to actually carry out the mitigations. These features will start being rolled out in summer 2024, with the full set available by November, says Dahan.

Fortinet’s FortiTester offers MITRE ATT&CK simulation testing, CVE-based IPS tests, and DDoS traffic generation. It can simulate various traffic types, including SSL, DDoS, and custom traffic, according to Expert Insights.

It also combines BAS with network performance testing, to make for a comprehensive tool. But it’s not as highly rated as some of the preceding tools by Gartner Peer Insights.

Mandiant is better known for its threat intelligence services, but it also offers breach and attack simulation via its Mandiant Advantage Security Validation software. Mandiant’s threat intelligence expertise is integrated into its BASE product, helping differentiate it from its competitors.

Its features include MITRE ATT&CK framework mapping, automated environmental drift detection and alerting, and real-world attack simulation.

NetSPI is best known as a penetration testing company, but it can also validate controls, identify detection gaps, and provide attack surface management by detecting potential vulnerabilities in public-facing assets.

The penetration testing expertise means that NetSPI customers can get some hands-on support, says Derek Wilson, NetSPI’s principal security consultant. “Our team of experienced pen testers will get on a call and screen share with your SOC team, walking through a subset of our procedures and taking notes about the detections and preventions they see.”

NetSPI is using generative AI to build a prioritized recommendation feature. “We will be able to use multiple data sources and quickly identify and prioritize the individual tests that would provide the end user with the most value,” Wilson says. Other potential uses of generative AI include dynamically generating playbooks based on the most recent threat intel for specific industries and building dynamic attack chains and helping to identify where there may not be enough coverage.

Picus Security is the second-highest rated BAS vendor according to Gartner Peer Reviews, and it was recognized as a “Customer’s Choice” for BAS in Gartner’s 2024 report. The company says it has hundreds of customers worldwide, including Mastercard and ING.

Picus Security Validation platform includes BAS as a core solution, but it also offers automated penetration testing and attack surface management, says Volkan Erturk, the company‘s cofounder and CTO. It also supports SOC optimization and cloud security posture management, he says.

The company is investing heavily in AI and offers a generative AI-powered security analyst that can offer insights to companies about their security posture. “By asking questions, users can instantly review the findings of security validation assessments and get tailored recommendations to prioritize and address exposures based on their organization’s threat profile and the latest cyber threat intelligence,“ Erturk says.

A generative AI-powered tool, it can intelligently map an organization’s SIEM rules to the MITRE ATT&CK Framework and provide AI-based suggestions to mitigate threat coverage and visibility gaps.

Redscan’s Breach and Attack Simulation specializes in managed detection and response and penetration testing. As a result, the company brings a hands-on approach to breach and attack simulations. It creates customized breach simulations for companies based on decades of incident response and testing expertise. Redscan also provides expert advice on next steps following a simulation.

ReliaQuest is known for its security operations platform, GreyMatter, which was named by Gartner as a “Customer’s Choice” vendor in summer 2023 in its managed detection and response category, with 94% of customers saying they’re willing to recommend it, with particular strength in the midsize enterprise segment.

GreyMatter Verify offers fully packaged and field-tested scenarios that security analysts can run to get results immediately. The vendor frequently updates attack scenarios based on the latest threat intelligence. It maps threat coverage to security frameworks such as MITRE ATT&CK, checks if existing security products are working, and can evaluate if new security controls will be effective.

Enterprises that don’t mind getting their BAS and their MDR from the same vendor might appreciate the integration.

SafeBreach: In Gartner’s Peer Insights, SafeBreach was the fourth best-rated dedicated BAS vendor. The company is known for its integration with other security tools and claims Netflix, PayPal, Pepsi and the Carlsberg Group among its customers.

SafeBreach’s BAS tests the efficacy of existing security controls with more than 25,000 attack methods from its Hacker’s Playbook. It claims to add new threats to the platform within 24 hours.

Customers can create customized attack simulations. It uses the MITRE ATT&CK framework and also provides a cost assessment of risk reduction efforts.

What to ask before buying BAS

Forrester’s Nost suggests that enterprises start their BAS journey by getting good visibility into the systems and controls that they actually have in place. “You don’t want to go out and get ahead of yourself and start using a BAS if you don’t know what you should be testing.”

Other questions to consider before selecting a vendor:

• How are they enabling detection improvements in their security controls with their products?
• Are they able to run testing at scale and in production with minimal impact on customer environments?
• What kind of research does the vendor conduct on the latest threats and adversaries?
• How frequently is the vendor’s threat library updated?
• Can you see an example report to see how simulation results are presented?
• Are the platforms open or only for black box testing? This means that the vendor tests are transparent so that you can understand what they are doing.
• Do regulators require on-premises or air-gapped deployments? If so, check that the BAS vendors offer those options — most only support SaaS.

Essential reading

• Penetration testing explained: How ethical hackers simulate attacks
• The changing role of the MITRE ATT&CK framework
• What is the cyber kill chain? A model for tracing cyberattacks
• Red vs. blue vs. purple teams: How to run an effective exercise
• How to prepare for an effective phishing attack simulation