CONTEXT

Covid Act Now built local data models and products to help governments, private organizations, and tens of millions of people make informed, science-based decisions on how to stay safe during the COVID pandemic.

When I joined the Act Now team in late 2020, Customer Support was a full-time role consisting of treating the symptoms of many problems (e.g., user confusion, data issues) without addressing their root causes. The Customer Support Lead would respond to users and answer their questions, but no one was documenting consistent themes coming into the email inboxes. Furthermore, users often were flagging data and product bugs, which were not being triaged.

Analysis

I coded each email on various characteristics, including User Type, Email Category, and Product with Issue.

Once all emails were coded, I ran counting functions and proportions for each coded category to see trends in email and user types. I then manually identified trends within each category (e.g., trends within the product request emails).

Reporting

Each week I would break down the findings from the inbox analysis in a detailed report document* that surfaced trends in email types and highlight impactful outreach (e.g., partnership requests, notable feedback). 

We would receive anywhere from 75 to 150+ emails per week, on average, typically fluctuating alongside COVID waves.

*Inbox report not included here, as it contains confidential information that is internal to the Act Now Project only.

Over time, the proportion of emails received that were about data bugs decreased by almost 50%, suggesting that the reports were associated with data quality improvements over time.

5th Color

At the beginning of the second wave of COVID-19 in the fall of 2020, many users emailed to express concern that the map was no longer useful-- with nearly all states in the red/high risk category, there was no way for them to tell the difference between states that might be just over the "high risk" threshold, and states experiencing even more severe outbreaks.

We added another risk level and corresponding map color (dark red) to add more nuance to these differences.

Vaccine Eligibility Alerts

Right after vaccines became available to the general public, our inbox flooded with users confused about when they were eligible and where to sign up when it was their turn.

We developed an email alert system that allowed users to subscribe to states of interest, and we then sent them emails whenever their state entered a new tier of eligibility. To remove another potential friction, we included a direct link to appointment booking websites so eligible parties could immediately sign up upon learning of their eligibility.

We also added a "Find a vaccine" link on every location page to make it easy and obvious to get an appointment booked, and in the places they already check on our site. This links to vaccines.gov, the government's official page for finding pharmacies with vaccines available near you.

Future research study recruitment

As part of another research effort, I wanted to conduct contextual inquiries to understand how our more in-depth users were utilizing our tools and data to make decisions for their respective groups (e.g., schools, workforces, community groups, etc). 

I was able to leverage the inbox analysis spreadsheet to easily pull email addresses of any users from commercial, government, healthcare, or other local organizations that had previously contacted us. I used this as my contact list to send out a call for participation.