Deploying Data-Driven Solutions for Public Health

As the Covid-19 pandemic descended on the world, many people working in public health found themselves building a response under constantly changing, highly unpredictable conditions. The County of Santa Clara Public Health Department was no exception. As with county public health departments across the state and country, every day seemed to require strategic tweaks or entirely new strategies while the virus churned through the population, hitting harder in some areas than others thanks to the vagaries of human behavior and sociopolitical factors. 

A public health task that was especially intense during the first year of the pandemic was contact tracing. For a new, life-threatening virus, tracing people in contact with an affected person is a critical measure. But success in this realm means working through a number of obstacles, from people’s natural reluctance to divulge health details to strangers to potential suspicion about government intrusion into private life. Adding a layer of complexity, it rapidly became clear that effective tracing also requires meeting the linguistic needs of potential contacts to ensure clear, accurate communication exchange.

Another task was ensuring that local residents had access to testing so that those with Covid-19 could seek healthcare, and that testing could be strategically deployed for especially vulnerable neighborhoods and households that would benefit from testing and social services resources. 

In the end, the keys to greater effectiveness in both cases were the same. One was a machine learning algorithm developed for the county's public health department by its partner, Stanford University’s Regulation, Evaluation, and Governance Lab (RegLab), a newly-announced recipient of Stage 3 funding from Stanford Impact Labs. The artificial intelligence (AI) solutions in turn guided the second key strategy: deployment of the human touch in the form of community health workers in local neighborhoods and efficient language matching between contract tracers and people testing positive for SARS-CoV-2.

The partnership with Santa Clara County is one of several that RegLab has established with government agencies, including other county public health departments and the Environmental Protection Agency (EPA). An interdisciplinary group of researchers from RegLab that includes data and social scientists works with government agencies to integrate state-of-the-art machine learning, AI, and statistical approaches into their processes and strategies. 

This partnership diverged from the usual experience of similar collabs with government, said Sarah Rudman, MD, MPH, deputy health officer and director of the county’s Infectious Disease and Response Branch. Many potential partners from academia and industry reached out to the county to offer to help, she said, but the typical partnerships with academia weren't a fit for the urgency of the moment. The success of such partnerships can be hindered if the partner goals – such as conducting long, painstakingly designed studies – don’t align with county resources. 

“We have a limited set of resources to solve major health problems, and our communities couldn’t afford to wait for us to conduct typical long-term studies,” Rudman said. In the case of RegLab, however, “they helped us make rapid operational decisions about the best ways to serve people with specific local needs. They listened to what we were doing and struggling with, and they had solutions we hadn’t even thought of, like the machine-learning model.” Even with the rapid response, the county and RegLab were able to conduct and publish studies that contribute important insights for public health planning.

An Emergent and Urgent Need

Early in the pandemic, Santa Clara County’s public health department joined four other Bay Area counties in issuing the first stay-at-home orders in the country. A few months later, in summer of 2020, RegLab partnered with the county’s public health team to sharpen and refine important strategies for reaching the county’s ~2 million residents, about a half million of whom are Latinx. In the chaos of attempting to respond to an encroaching virus that exposed massive inequities in U.S. healthcare and resource allocation, the county had to recruit scores of contact tracers to reach out to people testing positive for SARS-CoV-2 and interview them about their recent contacts. In addition, because of testing gaps in neighborhoods across the county, health workers also needed to get free Covid tests into underserved areas with highest vulnerability.

In the contact tracing piece, inefficiencies arose because the existing process would match Spanish-speaking tracers with people who did not need Spanish-language support, slowing contacts with Spanish-speaking community residents. The county needed a solution that better matched Spanish speakers on both sides of the contact encounter to mitigate this barrier. RegLab stepped in with a solution, developing an algorithm that supported more efficient matching. The result was a much shorter time for contact tracing and better communication from public health to vulnerable communities disproportionately affected by the pandemic. 

The algorithm relied on limited information from testing lab reports and more extended census data to streamline the matching process. In spite of the exigencies of the pandemic, county workers and their RegLab partners assessed the matching algorithm in a randomized controlled study that showed big effects: 

• Time from case initiation to completion dropped by almost 14 hours.
• Rates of completed outreach to all contacts for a specific case within one day increased by 12%. 
• With the comfort of speaking the same language, there was a 4% reduction in refusals of the contact tracing interview.

“Everyone remembers how important it was to be doing the best science and good work during the pandemic, and we were very happy to provide that kind of service,” said Derek Ouyang, RegLab’s research manager. “We demonstrated how academics can engage with local partners, how research should be done with an immediate public health need,” he said, noting that the work led to over a half dozen unique projects during the pandemic, including the partnership with Santa Clara County. 

The ongoing outcome of all of that in-the-moment effort is that county public health officials now have a toolkit in the event that contact tracing again becomes necessary, one with potential for other counties with similar needs. In addition, languages other than Spanish might be incorporated into the contact tracer–matching algorithm.

Machine learning also could be foundational in solving the problem of getting tests to under-resourced, vulnerable areas. During the height of the pandemic, community-based health workers in these neighborhoods went from door to door, offering the testing. The hitch was figuring out which neighborhoods in the county most needed access to this resource. 

Rudman views this work as having had the greatest impact among their RegLab-partnered projects, and the numbers bear out the important effect. When county health workers relied on confirmed case rates in local neighborhoods, the positivity rate with testing outreach was 2.6%. But when they relied on local knowledge about where cases were accumulating and community-health workers knocked on doors in those areas, positivity rates were 6.4%.

It may seem counterintuitive to view a greater rate of positive tests as better, but in this case, it meant identifying people with Covid who otherwise might not have been. Those who were asymptomatic could take precautions, and those who were ill could seek appropriate care. The community-health workers themselves were recruited through a local Latinx community organization, and in the end, they ultimately reached a higher proportion of Latinx individuals: 90% of those tested door-to-door versus 31% of those tested at the local fairground.

In a study reporting on this effort, published in JAMA Health Forum, RegLab staff describe an even more effective strategy: uncertainty sampling. This approach uses test positivity rates in an area, but takes into account the weaknesses of relying on this testing. For example, an area may present with a low positivity rate not because the virus isn’t in wide circulation but because of lack of access to and awareness of available testing. When RegLab built these elements of uncertainty into their model, they found that test positivity rates bounced up to 10.8%, well over the 2.6% rate from testing in areas with already known high case rates.

RegLab and Santa Clara County public health partnered on several other projects during the pandemic, including testing out the idea of using “high-touch” contact tracing to connect people testing positive to needed social services. The partnerships led to several published studies and garnered an innovation award from the National Association of County and City Health Officials.

Even if the tools developed specifically for the Covid-19 response don’t translate directly for other public health–related planning, Santa Clara County Public Health Department staff continue to use insights from the experience. Rudman highlights the promise of AI-based approaches and the importance of deriving solutions from local knowledge, as with the locally informed door-to-door testing, instead of relying only on bias-susceptible central data collection. 

With insights gained from the algorithm RegLab developed, “one of the most important aspects was to overcome historic mistrust in government and instead use this research and data to show what government can do to help,” she said. In this case, the result was “making access to health services more equitable and doing so in a way that has buy-in from the community. We are a big leap closer to using machine learning in other ways because of the experience.”

That’s the end goal for RegLab, said Ouyang, which seeks to step in where the government lacks resources to think about AI and machine learning, as with Santa Clara County during the pandemic. RegLab goes “where there is a real social problem and a way to leverage data more effectively and stay engaged deeply enough through implementation,” he said, “so that ultimately, the solution can manifest in government without our being involved in the long run.”

To learn more about RegLab's pandemic response work, read more here.