Dynamic businesses often require rapid hypothesis testing. At this stage, quick implementation is more important than stability and performance.
However, after confirming a hypothesis, it’s equally significant to make the solution stable and reliable. The business is already used to having this new feature around from the testing phase. So, you need to move fast and smooth when turning a prototype into a full-fledged solution.
The transition from an MVP to a live solution should be as seamless as possible. It’s like upgrading from a bicycle to a car – you want to keep moving, just more efficiently.
In the delivery services, keeping drivers happy is a constant battle. Our client was losing this battle, watching their drivers jump ship to competitors offering better perks. They needed a fresh approach – one that could quickly test new ideas, offer flexible pay, and give drivers instant feedback on their performance.
In this case study, we’ll break down our process of rapid hypothesis testing and show how it benefited both the delivery service and its drivers.
We took on the client’s issue head-on with a methodology centered on rapid hypothesis testing. Our strategy wasn’t about grand, time-consuming overhauls. Instead, we focused on swift, targeted experiments. This approach allowed us to:
We developed a process that allowed us to move quickly while still gathering meaningful data. Here are the steps that form the backbone of the approach:
We worked closely with the client to identify potential solutions to their driver retention problem. Each idea was framed as a testable hypothesis.
Our team quickly developed basic versions of each proposed solution. These prototypes were functional enough to test the core idea without requiring extensive development time.
We monitored key metrics closely during each test period. Our team used both quantitative data (e.g., retention rates, driver activity levels) and qualitative feedback from drivers.
Based on the data collected, we made swift decisions about whether to pursue, modify, or discard each hypothesis.
Hypotheses that showed promise were later scaled up and integrated into the main system.
Our quick-fire approach to testing ideas wasn’t just fast – it proved effective across various aspects of our project:
⏳ Time efficiency: By testing multiple ideas one-by-one or/and simultaneously and making quick decisions, we reduced the time from idea conception to implementation. This agility allowed us to stay ahead of market trends and competitor offerings.
🗑️ Resource optimization: We focused our resources exclusively on ideas that demonstrated genuine potential. This targeted approach minimized waste and maximized the return on investment for our client.
🚕 Adaptability: The rapid testing cycle allowed us to quickly adapt to changing market conditions and driver preferences.
📈 Continuous improvement: Even after implementation, we continued to refine and optimize solutions based on ongoing feedback and data.
📋 Risk mitigation: We implemented manual verification of automatically calculated results. This additional layer of oversight helped catch potential errors early, ensuring the accuracy and reliability of our incentive systems.
The existing system offered only one tariff for all drivers, limiting our ability to motivate performance effectively.
💡 Hypothesis: Different driver groups require tailored tariff structures.
🔧 Implementation:
✅ Outcome: The flexible system improved driver satisfaction across different groups, providing a solid foundation for later experiments.
Hypothesis: Implementing daily bonuses for more active drivers will increase overall driver engagement and productivity.
Implementation:
✅ Outcome: The implementation of daily performance bonuses proved highly successful.
💡 Hypothesis: Offering two potential tariffs per driver would optimize earnings and increase motivation.
🔧 Implementation:
✅ Outcome: The dual tariff system proved to be a significant success:
💡 Hypothesis: Introducing additional weekly motivational bonuses would further enhance driver engagement and retention.
🔧 Implementation:
❌ Outcome: This idea did not progress beyond the prototype stage. The weekly bonuses did not provide the expected boost in driver engagement or retention
While this hypothesis did not show the desired results, it highlighted the value of our rapid testing approach. By quickly prototyping and testing the idea, we avoided significant investment in a full-scale implementation of an ineffective strategy
After testing the remaining hypotheses, we identified the optimal set of tariff parameters and confirmed the accuracy of our calculations.
With confidence in our solution, the client decided to transition to a live implementation. This move meant the elimination of manual verification stages for both calculations and transaction uploads. The new system was designed to automatically create transactions in the production environment immediately after calculations were completed
While removing the verification step was easy, we faced a new challenge: the need for real-time data display for drivers:
This led us to tackle two critical aspects of our solution: performance optimization and user interface implementation.
Our temporary solution, built on RedShift, faced significant limitations. It couldn’t meet the demand for 500 simultaneous connections, far exceeding its 100-connection limit. We also needed to ensure that data updates occurred within 3 seconds of order completion, allowing drivers to see their income immediately.
To address these issues, we transitioned to PostgreSQL, leveraging our team’s expertise with this database system. This shift helped us optimize requests for data retrieval, calculations, and transaction loading into the production system.
PostgreSQL’s scalability also enabled us to handle unlimited simultaneous connections and requests.
For the user interface, we had to innovate within constraints. The existing app lacked any income display functionality, and developing a native solution was both costly and conflicted with other priorities.
Our solution was to implement a Web-view in the Driver App, a technology we had successfully used before. This approach allowed us to rapidly deploy the new feature and, as an added benefit, integrate an identical view in the Admin Panel for support agents. They also needed this feature as they get a lot of requests about the drivers’ payments daily
As a result, the client got a consistent interface for both drivers and support staff, improving overall service efficiency.
Our agile hypothesis testing approach enabled the client to rapidly iterate and find the most effective driver retention strategies.
Just in three months, the project progressed from a Google Sheet prototype to a live system used by thousands of drivers daily.
After we implemented the display of earnings and KPI in the drivers’ application, the drivers were so delighted with this feature that they promoted our delivery service to other drivers. At that moment, driver retention was at its peak, and attracting new drivers happened organically through word of mouth.
