Can digital medicine build evidence less like Prozac and more like Pinterest?

The Future of Healthcare

Mental models are a powerful force.

When the emperor arrived on the streets in his “new clothes,” it wasn’t so much his foolishness that the tailor was taking advantage of as it was the emperor’s unshakable mental model of tailors, clothes, and status. The emperor had such a strong belief that clothes would emerge from the tailor’s process (measuring, sewing, fitting) that he allowed himself to be dressed in nothing, based on the power of that belief alone.

Indeed, mental models can be incredibly powerful and useful tools to align ideas and drive progress. But they can also be powerful blockers of vision.

At this moment in time, I wonder if digital health is being held back by the mental model of the phased clinical trial system for the creation of evidence. Fundamentally, it’s a fine system for molecule-based treatments like Prozac to go through, but you can’t do every job with the same tool. By focusing evidence creation on large phased and highly regulated trials, we’re missing huge opportunities to take advantage of the unique, fast-paced, continuous feedback cycles made possible in tech products.

In a blog post from 2017, an engineer at Pinterest wrote, “Every day we aim to have experiment results ready by 10 a.m. so we can make fast and well-grounded decisions. With more than 1,000 experiments running daily, crunching billions of records for more than 175 million Pinners, we need a reliable pipeline to support our growth…” My PhD thesis, on the other hand, had taken five years and included maybe half a dozen experiments.
For me, hearing about Pinterest’s experimental capabilities for the first time was revelatory. What they have is a rigorous protocol system for fully controlled (yes, controlled!), scalable, modifiable, real-time experiments. The protocol is iterative and accessible, too; there is a clear process for anyone on the product team to submit product experiments that they want to validate for ongoing development.

This means that when Pinterest makes a product change, it’s able to do so by implementing tiny changes already proven within small controlled experiments to lead to product growth and/or a better user experience.

From a design perspective this is very cool, and from a research perspective, it’s staggering. We are also called to wonder: what if all this evidence design and experimental power could be translated to a product that, instead of focusing on curating photos of artisanal cupcakes focused instead on, say, treating mental illness?

The core feature of the clinical trial system is that it ensures safety and mitigates risk. What the process doesn’t do well is enhance benefit. At the end of a trial all you know is, did black box molecule X do better than sugar pill Y? You know nothing about the mechanism of action or how to make it better next time. The model for Pintrest, on the other hand, is nearly exclusively about continuous improvements. The experiments are much smaller and shorter, the feedback cycle is tight, and the data is accessible for future thought and revisions.

The kind of evidence creation driving Pinterest is undoubtedly complex and the mapping to digital health products is not straightforward (increasing click traffic is a more direct goal than decreasing loneliness, for example). But the first thing we can shift is our mental model of how evidence can be generated.

How can we think more continuously about the evidence we create? How can we generate more, smaller improvements faster? and how can we ensure safety in this new way of working?

Stephanie Greer, Hopelab’s former Research Lead, is a curious and creative thinker with a diverse skill set in behavioral neuroscience, design, and digital product development. Prior to Hopelab, Stephanie earned her Ph.D. in neuroscience studying health decision making (including sleep and dieting behavior) and was an early member of the Health Special Projects team at Apple. She is currently on sabbatical, gleaning knowledge from cultures around the world. 

Related Content

View all Insights
amy and jaspal headshots
The Future of Healthcare

We welcome two new additions to the Hopelab team: Jaspal S. Sandhu, Ph.D., as Executive Vice President (EVP), and Amy Green, Ph.D., as Head of Research.

Read More

A panel of experts weigh in on recent legislation harming transgender and LGBTQ+ youth and share guidance for how we might offer support.

Read More
purple background with four images and text saying "A shared set of experiences: Black youth, anti-black racism, and adolescent development"
Health Equity

In conversation with Hopelab, Jenn and Joanna talk about the mission of the NSCA, and dive into why anti-Black racism specifically needs to be highlighted in the understanding of adolescent development, the opportunities afforded by social tech, and how we translate knowledge to action.

Read More
Julie Tinker sits on a couch looking at the camera with her arm propped on up
Health Equity

As we strengthen our systems of equity and accountability within our teams and across our partnerships and products, we can’t think of anyone better to join us at the forefront than Julie to help foster a culture of learning, growth, opportunity, and joy.

Read More