Two game-focused Khoury researchers are turning amateurs into scientists

It was extraordinary, but you had to look closely to spot it.

Tucked into the June 20, 2019 issue of Nature was a paper titled “De novo protein design by citizen scientists.” Among its 21 authors was one credited as “Foldit Players.” And if you were wondering what sort of hipster parents would name a child that, it turns out something far more fascinating was going on.

Those researchers earned the credit by playing Foldit, an online game in which they fold proteins into new and more efficient shapes. In July, the structures they built were featured in a popular feature by Protein Data Bank, a database of protein structures used by researchers.

“The players were really excited to be Molecule of the Month,” said Seth Cooper, assistant professor at the Khoury College of Computer Sciences  and Foldit’s original lead designer. “Seeing the players involved in other aspects of the scientific process and get recognized for the work—or play—that they did, that’s really fun and exciting.”

If Cooper and his doctoral student, Josh Aaron Miller, have their way, it won’t be the last time it happens. Through their continued work on Foldit and on scientific discovery games, the pair aim to turbocharge scientific discovery by harnessing and developing the creativity and knowledge of the gaming world. And they know it has educational potential, too.

“Games tend to provide experiential learning, which Northeastern is huge on,” Miller noted. “You’re interacting with the medium, you’re experiencing it, and that opens up a lot of potential for active learning.”

Crowdsourcing science through video games
As a scientific discovery game, Foldit falls into the larger category of citizen science, in which the contributions of amateur scientists are crowdsourced and combined with those of professionals. It’s as fertile a ground as any for innovation and unconventional research methods, and it offered a natural path for a budding computer scientist inspired by his childhood love of video games.

“As an undergrad I got into computer graphics, interactive simulations,” Cooper recalled. “Then I went to grad school, and in a way, computer graphics was as close as you could get to game-related things in an academic setting at that time.”

At the University of Washington, Foldit presented itself as an ideal doctoral opportunity. The idea, Cooper said, “was to take human reasoning, problem-solving, and spatial abilities, then apply them to solving challenging computational problems.”

What they produced was a platform to which professional scientists uploaded unfinished folding puzzles, which players manipulated following rules for stabilizing interactions. The computer would refine the fit, then researchers would examine the best solutions, looking for configurations they could apply to real proteins. The result, ideally, would be scientific innovation.

Getting there required threading multiple needles. Foldit needed to be interesting, fun, and accurate enough that player contributions would be scientifically useful. It also, Cooper noted, required a method of quickly translating biologically sound structures into scores to entice players. Do that, and they could unlock the educational and scientific potential unique to games.

“The instructional method is responsible for how much you learn and how much you’re motivated to learn,” Miller explained. “But the power of the medium is the efficiency of delivering that, the ease of access, and the cost of doing that instruction.”

And Miller knew from experience just how fluid game-related learning could be. He recalled taking an undergraduate neuroscience course where the presentation style made otherwise understandable material tricky to absorb.

Foldit has succeeded beyond its makers’ wildest dreams, attracting hundreds of thousands of players across its 13 years. Cooper attributes the success partly to players sharing strategies, collaborating, and discussing the game on forums. Miller recently won a Rosetta Service Award—awarded by the group that created the modeling software Foldit is based on—for his community building.

“I created a Discord for Foldit which now has over 2200 members,” Miller said. “In July 2020, I started writing a weekly newsletter and sending it to anyone who wanted to subscribe. We have about 100 subscribers, plus more read it once it’s posted to the forums a week later. The newsletter shares weekly tips and the top solutions players come up with.”

Some of those top solutions have turned heads. A handful have matched or bested algorithmically computed solutions and player efforts could help to improve the algorithms. Players have created 56 unique synthetic proteins from scratch. And, in arguably the most remarkable case, scientists turned to Foldit players for help solving the structure of a protein that causes AIDS in rhesus monkeys. An accurate shape would have given scientists a better chance at designing antiretroviral drugs.

The scientists, using traditional methods, had worked for more than ten years without solving the problem. Foldit players solved it in ten days and when it came time for research credit, the players who spearheaded the solve, true to citizen science form, asked that they be credited not as individuals, but by their Foldit team name.

The future of the field
In 2020, Google’s DeepMind debuted AlphaFold 2, an artificial intelligence program that predicts protein folding. Co-founder Demis Hassabis noted that Foldit inspired him to wonder if AI could mimic the gamers’ intuitive capability.

“AlphaFold 2 is a big deal in terms of the quality of the structures it produces compared to previous methods,” Cooper said. “Machine learning has been increasingly looked at in bioinformatics and protein structure prediction in the past few years.”

But Cooper also noted that there is still room for AlphaFold to progress, and that Foldit has spent the last few years moving from protein structure prediction—what AlphaFold does—toward the design of novel proteins. And as long as there’s a role for Foldit, Cooper and Miller are resolved to continue improving it.

The tutorial is a huge focus of this effort, and when Miller arrived at Khoury College in 2017, he went to work on it. It’s a three-pronged approach, he said, in which the tutorial makes a positive impression on new players, improves the odds of retaining them, and helps them understand the game and its goals.

“To make a useful contribution, you need to understand what you’re doing and build up expertise,” Miller said, before pivoting to Pixar’s Ratatouille as an example. “‘Anyone can cook’ means a great cook can come from anywhere. To play these games, you don’t need to know the biochemistry. But by playing, you intuitively understand the rules of the biochemistry, learn your own expertise, and learn to fold proteins based on that … I’m researching how to make that learning easier, how to onboard people better. Because sometimes it can take years for people to get enough expertise to make meaningful contributions. We want to shorten that gap.”

For his doctorate, Miller plans to overhaul the Foldit tutorial using the four-component instructional design model. He’ll conduct a skill-based cognitive task analysis of how users play and what the experts do, then develop a curriculum teaching new players to understand the game like the experts.

Cooper added that they are trying to leverage player data to make the tutorial levels more interesting, possibly by customizing their difficulty to each player’s demonstrated skill level.

“A lot of people come in, look around a bit, and leave,” he noted. “So, we’re trying to help new players get involved and not get discouraged by the relatively challenging problems they have to work up to.”

They also want to overcome the barriers that confine citizen science games to what Miller called a “quiet indie space.”

“Developing citizen science games is a strongly interdisciplinary practice; it requires a lot of expertise from a lot of distinct fields,” he explained. “You need scientists working on it, but you also need professional software developers, game designers, producers, marketers, journalists, community managers, QA testers.”

Combine that with the differences in funding models—grants for science, sales for game studios—and you get underfunded, understaffed citizen science games.

“They suffer in several ways: unfinished game design, unpolished gamification, software issues, technical issues, bugs, crashes,” Miller said. “The scientists don’t have time to communicate, and they’re not working with science journalists who can help them describe what’s happening in a way that’s accessible.”

But Cooper and Miller remain hopeful for the field’s future. Each pointed to the inclusion of citizen science components in mainstream blockbuster games—Miller named EVE Online and Borderlands 3—as evidence that something bigger could be on the horizon. And Cooper cited similar citizen science games dealing with neuron tracing, RNA folding, genomic sequence alignment, and image categorization as standouts.

And they know just how valuable it will be for scientists to bring quality game designers into the fold.

“I don’t think making something a game inherently makes it easier or more fun to learn,” Miller says. “It really is about the execution.”

Subscribe to the Khoury College newsletter