We are all subjected to recommender systems everyday, from newsfeed algorithms to ecommerce recommendations or automated playlists. They are famously opaque, despite their (likely) enormous influence on our lives.

Data journalists and others regularly attempt to shed light on their inner workings. The main finding so far is sobering: no method currently exist to effectively assess the effects of a recommender system.

My presentation will provide the audience with a structured and documented overview of these efforts in the last 10 years, in Germany and beyond. Based on many examples from newsrooms, regulators and academia, I will explain what has been attempted, in terms of experimental set-ups, methods, risks – and results. I will end with a hopeful note by sharing latest consensus on platform auditing: we need to use a wide variety of methods, from ethnography to computer science and traditional shoe-and-leather journalism to stand a chance.