This Algorithm Could Ruin Your Life

From the outside, Rotterdam’s welfare algorithm appears complex. The system, which was originally developed by consulting firm Accenture before the city took over development in 2018, is trained on data collected by Rotterdam’s welfare department. It assigns people risk scores based on 315 factors. Some are objective facts, such as age or gender identity. Others, such as a person’s appearance or how outgoing they are, are subjective and based on the judgment of social workers.

In Hoek van Holland, a town to the west of Rotterdam that is administratively part of the city, Pepita Ceelie is trying to understand how the algorithm ranked her as high risk. Ceelie is 61 years old, heavily tattooed, and has a bright pink buzz cut. She likes to speak English and gets to the point quickly. For the past 10 years, she has lived with chronic back pain and exhaustion, and she uses a mobility scooter whenever she leaves the house. 

Ceelie has been investigated twice by Rotterdam’s welfare fraud team, first in 2015 and again in 2021. Both times investigators found no wrongdoing. In the most recent case, she was selected for investigation by the city’s risk-scoring algorithm. Ceelie says she had to explain to investigators why her brother sent her €150 ($180) for her sixtieth birthday, and that it took more than five months for them to close the case.

Sitting in her blocky, 1950s house, which is decorated with photographs of her garden, Ceelie taps away at a laptop. She’s entering her details into a reconstruction of Rotterdam’s welfare risk-scoring system created as part of this investigation. The user interface, built on top of the city’s algorithm and data, demonstrates how Ceelie’s risk score was calculated—and suggests which factors could have led to her being investigated for fraud.

All 315 factors of the risk-scoring system are initially set to describe an imaginary person with “average” values in the data set. When Ceelie personalizes the system with her own details, her score begins to change. She starts at a default score of 0.3483—the closer to 1 a person’s score is, the more they are considered a high fraud risk. When she tells the system that she doesn’t have a plan in place to find work, the score rises (0.4174). It drops when she enters that she has lived in her home for 20 years (0.3891). Living outside of central Rotterdam pushes it back above 0.4. 

Switching her gender from male to female pushes her score to 0.5123. “This is crazy,” Ceelie says. Even though her adult son does not live with her, his existence, to the algorithm, makes her more likely to commit welfare fraud. “What does he have to do with this?” she says. Ceelie’s divorce raises her risk score again, and she ends with a score of 0.643: high risk, according to Rotterdam’s system.

“They don’t know me, I’m not a number,” Ceelie says. “I’m a human being.” After two welfare fraud investigations, Ceelie has become angry with the system. “They’ve only opposed me, pulled me down to suicidal thoughts,” she says. Throughout her investigations, she has heard other people’s stories, turning to a Facebook support group set up for people having problems with the Netherlands’ welfare system. Ceelie says people have lost benefits for minor infractions, like not reporting grocery payments or money received from their parents.

“There are a lot of things that are not very clear for people when they get welfare,” says Jacqueline Nieuwstraten, a lawyer who has handled dozens of appeals against Rotterdam’s welfare penalties. She says the system has been quick to punish people and that investigators fail to properly consider individual circumstances.

The Netherlands takes a tough stance on welfare fraud, encouraged by populist right-wing politicians. And of all the country’s regions, Rotterdam cracks down on welfare fraud the hardest. Of the approximately 30,000 people who receive benefits from the city each year, around a thousand are investigated after being flagged by the city’s algorithm. In total, Rotterdam investigates up to 6,000 people annually to check if their payments are correct. In 2019, Rotterdam issued 2,400 benefits penalties, which can include fines and cutting people’s benefits completely. In 2022 almost a quarter of the appeals that reached the country’s highest court came from Rotterdam.