Low SMS Deliverability: Root Cause Analysis

While interviewing with Uber for a role as a Product Manager, I proposed a solution to low SMS deliverability, commonly plaguing the Application-to-Person (A2P) messaging space.

Product Manager Candidate | Uber Communications Platform | November 2021

Problem Statement

The following is a paraphrased case prompt provided by Uber.

SMS messages, and particularly one-time passwords (OTPs), are a critical part of the Uber user experience, necessary for phone number verification during new account creation as well as retrieving lost or forgotten credentials. Two KPIs that Uber prioritizes in this space are cost in USD and overall deliverability, i.e., rate at which SMS is successfully delivered to customers.

The SMS ecosystem is complex. It consists of different SMS providers in different regions, with each provider having independent aggregators through which SMS is routed.

Over the past month, there has been a significant decrease (> 5%) in SMS deliverability, which is capable of generating considerable friction for users looking to sign up or sign in to Uber. How should this issue be diagnosed, root causes be identified, and a solution to stem the impact to users and the business be crafted and executed?

Presented with this open-ended prompt, I followed a user-centric design process to investigate and eventually pitch a plan. Aware that any solution begins with understanding a problem, I first weighed potential causes of OTP SMS delivery failure, impacting thousands of active users of Uber Eats and Uber Rides globally. Then, priority and impact guided my final recommendation.

Design Process

In conducting my Root Cause Analysis and recommending a concrete path forward, I followed this sequence of steps:

  1. Define Problem

  2. Gather Missing Information

  3. Establish Business Impact + Success Criteria

  4. Identify Shortcomings in Input Quality

  5. Explore Feasibility and Impact of Solution Alternatives

  6. Recommend Best Option with Anticipated Outcomes

  7. Plan Execution

Research Methods

To understand the hit that SMS deliverability —particularly, OTP SMS— took, I investigated common demands and pitfalls of A2P SMS. With no prior experience in A2P messaging but plenty of experience communicating asynchronously with my international students, I had observed the impact of splintered communication. I funneled this recognition into my research, which took the form of empathetic in-person interviews and independent study of industry-wide trends.

Additionally, owning an investor-facing chatbot at BXTI had instilled in me that:

  1. Partnership with external vendors requires both transparency and responsible withholding of sensitive information. SaaS vendors offer a breadth of data resulting from exposure to multiple clients, which can either accelerate or risk internal business goals.

  2. Users receiving communications from applications or websites respond well to expected, consistent messages and poorly to unexpected or redundant ones.

These principles contributed to my final suggestion to introduce an SMS Quality Control System to champion A2P communication, specifically via SMS, across Uber.

Championing Uber’s A2P SMS uncovered some strategies for my own long-distance interactions, too!

My presentation detailing my findings and ultimate recommendation is below:

A infographic about Uber's SMS deliverability decline, with two main sections: data requirements and success criteria. The left section in blue states that Uber's SMS deliverability has fallen significantly over the past month, impacting customer acc
Text document titled 'Setting the Scene' with sections on feature vision and goal, business impact, SMS ecosystem, and key performance indicators, discussing Uber's SMS messaging service.
Infographic listing potential problem areas affecting SMS delivery, including device issues, carrier filters, routing through grey routes, and high network traffic.
A diagram explaining the benefits, drawbacks, and complexity of introducing an authenticator app to replace SMS for verification, with text boxes connected by lines.
Page from a presentation discussing integrating external security tools and vendors, featuring sections on benefits, drawbacks, and complexity of the approach.
A presentation slide detailing a plan to develop a robust SMS quality control system for Uber. The slide includes sections on benefits, drawbacks, and complexity of the proposed solution, with text organized in a clean, minimalist layout.
Screenshot of a presentation slide titled '[Final Recommendation] SMS Quality Control System'. It shows an example of a smartphone screen with SMS messages containing various Uber verification codes, along with blue annotations describing issues such
Flowchart titled 'Path to Execution: SMS Quality Control System' outlining steps like internal and external communication, requirements gathering, data analysis, feedback sessions, KPI measurement, email and push notifications, and automation integra

Outcomes

My recommendation to implement an SMS Quality Control System underscored my commitment to consistency and predictability: both on behalf of Uber’s outgoing SMS and in personal pursuits. I got the job! I know a lot more about A2P SMS today than I once did.

Still, quality control is not the most effective solution to deliverability drops in the SMS space. Uber leans on quality checks and last-mile rules to regulate email traffic, but a smart-routing system is an effective countermeasure against periods of low SMS deliverability. Such a system dynamically allocates high-priority SMS to vendors that exhibit strong deliverability over a specified look-back period of time, minimizing likelihood and impact of low deliverability.

My interviewers were kind to encourage me to factor vendor costs into future spars with low SMS deliverability. In 2021, I had framed my analysis purely in user- and deployment-oriented contexts, but now, I expect SMS costs to influence decisions around deliverability quite heavily.