ExploitGym

ExploitGym is a practice environment for computer programs that are meant to detect security vulnerabilities in software themselves. A security vulnerability is a programming flaw that an attacker can use to make a program do something it isn’t supposed to do. ExploitGym provides many such flawed practice programs, deliberately built to be vulnerable. An attack attempt takes place in an isolated test chamber on the machine, that is, separated from the real system. The environment then automatically checks whether the attempt succeeded and awards points. The name alludes to a fitness gym: many stations, clear measurements, repetition until improvement.

Why security research needs a benchmark

Almost every major company today claims its AI can find software vulnerabilities. Such claims are hard to verify. Without standardized tasks, one ends up comparing apples to oranges. This is precisely the gap that environments like ExploitGym fill: the same tasks, the same rules, comparable numbers.

The second reason is economic. Searching for security vulnerabilities is expensive manual work done by highly specialized experts. If software can take over part of this work, companies could review far more code than they do today. For investors, this is a billion-dollar market; for attackers, a new toolbox.

This also names the flip side. A system that can find vulnerabilities can also abuse them. Operators of such environments therefore usually limit the tasks to artificial practice programs rather than current real-world software. This trade-off is still debated controversially in the field today.

Task, sandbox, and scoring

A task consists of a small program with a built-in flaw and a goal. A typical setup is a hidden file whose contents only become visible through a successful attack. In the community, this content is called a flag. Whoever can present the flag has demonstrably solved the task.

The program being tested runs in a sandbox. This refers to an isolated area that behaves like its own small computer but is wiped after each attempt. Within it, the test subject may enter commands, read files, and crash the program. Nothing escapes to the outside. This protects both the test system and the validity of the measurement.

What is evaluated is not the explanation but the result. Either the flag is produced or it is not. In addition, secondary metrics are often counted: number of attempts, computing time used, cost. This makes it visible whether a system works in a targeted way or merely tries things for a long time. A common mistake is to confuse high scores with genuine attack capability. Practice programs are much smaller and tidier than real software with millions of lines of code.

ExploitGym in model reports and safety debates

The term is most often encountered in accompanying reports for new AI models. There, providers list how many tasks from such environments their model solved. These figures then find their way into press releases and market commentary. Anyone reading them should always ask exactly which tasks were counted.

A second setting is debates about regulation. Authorities and industry associations are looking for measurement methods to detect dangerous capabilities of AI systems early. Results from environments of this kind are considered an early warning sign. They are now being incorporated into voluntary safety evaluations by major providers.

ExploitGym should not be confused with classic hacking learning platforms for humans. Those are aimed at learners and explain solution paths. ExploitGym, by contrast, is designed for automatic evaluation, without pedagogy, with machine-readable results. Related to it are general training environments for software agents, in which programs, for instance, fix bugs in code rather than exploit vulnerabilities.