Automated Decision-Making System

An Automated Decision-Making System — ADMS for short — is software that independently makes decisions or at least issues a recommendation that people act on almost automatically. A human sets the rules by which the system judges. After that, it runs on its own. The outcome — approval, rejection, classification — comes out without any further human review. Such systems often process a great many data points simultaneously, from account transactions to click behavior in an app.

What’s at stake

ADMS make decisions that directly change the lives of individual people. An algorithm calculates whether a bank grants a loan. Another system filters out applications before a hiring manager has even glanced at them. Those rejected by the system often don’t learn why.

This creates a power problem. In the past, a caseworker justified their decision — at least in principle. An algorithm gives no justification, it gives a result. If the system systematically disadvantages certain groups, for instance because it learned from historically biased data, this error can repeat itself millions of times over without anyone noticing.

This is precisely why the European General Data Protection Regulation (GDPR) mandates that people have the right, in certain cases, to demand human review of automated decisions. The EU AI Act, which has been coming into force gradually since 2024, goes even further and classifies ADMS in particularly sensitive areas as “high-risk AI.”

Rule-based or learning — two fundamentally different designs

There are two basic ways to build an ADMS. The first way is fixed rules: If income is below X and debt is above Y, then reject. A human writes these rules down. They are traceable, but rigid — exceptions are hard to accommodate.

The second way uses machine learning, a method in which software works out patterns from sample data on its own. The model learns, for example, from thousands of past loan cases which characteristics correlate with default. It then decides without explicit rules — which makes it more powerful, but harder to see through.

This second type is often called a “black box”: you see what goes in and what comes out, but not how the decision is arrived at inside. A dedicated research field called Explainable AI aims to solve exactly this problem and make the decision paths of such models more visible.

ADMS in the news and everyday life

ADMS are encountered more often than one might think. Someone who gets a movie suggested on a streaming platform experiences a harmless variant. Someone applying for a rental lease whose creditworthiness is assessed via a credit score experiences a consequential one. In the US, such systems pre-sort résumés at large corporations, determine the amount of bail payments in court, and calculate the recidivism probability of offenders.

In financial reporting, the term comes up when banks automate their lending or when insurers start calculating premiums algorithmically. It is also present in political debate: in the Netherlands, a court declared a state ADMS for fraud detection illegal in 2020 because it was opaque and discriminatory — a ruling that drew attention across Europe.

The term is no longer a purely technical concept. It has become a legal and political battle cry, raising the question of how much decision-making power a society wants to hand over to algorithms.