‘Tip of the iceberg’: Elon Musk plans to liberate Twitter’s algorithm

“The people inside Twitter want to understand how their algorithm works, too,” he said.

Twitter, Facebook and Musk didn’t respond to requests for comment.

The opaqueness of algorithms that fuel what people see on social media sites has long been a fascination for conservatives, in particular, who have alleged without evidence that the platforms are biased against them. Musk’s attempts to gain sway over Twitter this week have resulted in rejoicing on the right, as some speculate that he may try to reduce the policing of misinformation and restore former president Donald Trump to the platform.

There have been efforts to make algorithms more transparent through regulation. Last year, several bills were introduced or reintroduced in Congress that focused explicitly on the software programs that decide what people see on social media platforms. Efforts are also advancing to regulate artificial intelligence and algorithms globally.

When social networks such as Facebook and Twitter were brand new, there wasn’t enough content to warrant need for a complex algorithm, which is essentially a set of rules similar to a mathematical equation that help parse through content and determine what is most relevant to an individual user.

But as hundreds of millions of users began joining and posting billions of pieces of content, the companies began writing software that could learn what users were most likely to click on and then sort their feeds accordingly.

Now, companies such as Facebook, video app TikTok and Twitter all use some form of algorithm to figure out what to show users. And it means that not every user on a social network sees the same thing. People obsessed with the outdoors might see posts about the best camping spots for the summer, while a basketball fan might be bombarded with posts about the NBA playoffs.

Facebook and Twitter initially had chronological feeds, and both have faced pushes recently to bring them back as a default, especially amid criticism that their ranking systems help promote the spread of misinformation.

The complex math that most companies use is often called “machine learning,” which is at its core a very fancy form of pattern recognition. A computer program can’t tell whether a particular tweet is funny, interesting or valuable. But if it looks at millions of tweets and a bunch of factors, such as who liked it, shared it and retweeted it, it can begin to predict which tweets are likely to get attention.

Even if Twitter made its secret formula public, including the math it uses to “train” its machine learning algorithms, an outsider looking at it wouldn’t be able to make meaningful conclusions from it, said Michael Ekstrand, an assistant professor of computer science at Boise State who researches recommendation engines.

An outsider would also need to see the underlying data used to “train” those algorithms – the billions of bits of data showing who looked at, liked or shared tweets, among many other possible factors.

Releasing that data would raise serious privacy concerns, experts said.

“The algorithm is just the tip of the iceberg,” said Robin Burke, a professor of information science at University of Colorado at Boulder and a researcher specialising in recommendation algorithms. “The rest of the iceberg is all of this data that Twitter has,” he said, most of which it can’t release publicly.

Musk should be well-versed in the complicated nature of algorithms, the experts said. Tesla, a company he runs, is using machine-learning algorithms to develop autonomous driving technology. That undertaking is so massive that Tesla is building its own supercomputer and customised semiconductors to process all the data.

There are other ways of improving transparency that are more practical, the experts said, some of which Twitter is already doing. Some critics have called on social media companies to simplify their algorithms so that when there are issues, like perceived biases toward specific groups of people, they can be more easily addressed. Others have called for independent audits inside the companies.

Twitter has an in-house research team called Machine Learning Ethics, Transparency and Accountability that looks at potential biases in its algorithms. It has published research, for instance, on whether the algorithms that automatically crop profile photos contained inadvertent bias.

Even if Musk succeeds and the code is released publicly, it’s likely critics will continue to be sceptical of whether it’s the full code and if something could have been left out.

One idea that has been floated is separating Twitter into multiple different algorithms. Twitter could select business partners that would have access to the data and develop algorithms tailored to certain audiences.

Nathan Baschez, co-founder of Every, a tech-focused writer’s collective, argued in an article Friday that Twitter should allow outsiders to create their own algorithms tailored to specific interests. Twitter co-founder Jack Dorsey replied to a tweet posted by Baschez, confirming that he had floated that idea while he was chief executive of the company before he stepped down in December.

In congressional testimony in November 2020, Dorsey suggested the idea broadly. Algorithms “are responsible for showing us what we see or what we don’t see and there needs to be more choice in their use,” he said. Twitter is also funding Bluesky, which aims to decentralise social media in part to give people more choice in how their feeds are organised.

A former Twitter employee who spoke on the condition of anonymity to discuss private matters said the company has considered an “algorithm marketplace,” in which users can choose different ways to view their feeds. But efforts to offer more transparency have proved challenging, the person said, because of how tied Twitter’s algorithms are to other parts of the product. Opening it up could reveal trade secrets and invite abuse, the person said.

Loading

Burke said that the idea has merit but would require a restructuring of the way Twitter operates and how its data flows. “The fact that it’s hard to imagine it happening points to the monolithic nature of these social media companies,” he said.

Releasing Twitter’s code to the public could also have negative side effects. Those aiming to game the system by spreading disinformation to sway an election could use the information to manipulate the platform, experts say.

Even if Musk succeeds and the code is released publicly, it’s likely critics will continue to be sceptical of whether it’s the full code and if something could have been left out. In essence, the billionaire would be asking Twitter users to trust him that there was no foul play going on behind the scenes, just as the company is today under its current ownership.

“It’s probably impossible to eliminate all scepticism and cynicism. There are always going to be people who don’t believe what’s being said,” Ekstrand said.

The Washington Post’s Rachel Lerman contributed to this report.

The Washington Post