Skip to content
← All work

Lifecycle · CRM · Experimentation

Holdout-tested email and push that measurably lifted retention

Took email and push from weekly manual sends to self-serve flows in MoEngage, then proved what worked with holdout groups, not guesswork.

+4.7%
Claims within 36h
+2.8%
Day-7 retention
2.4%
Email CTR vs 1.45% benchmark

At a glance

The problem
Every send needed a developer, and nobody knew whether a message changed anything.
What I did
Moved us to self-serve MoEngage, built trigger flows around key moments, and measured them against holdout groups.
The outcome
+4.7% claims within 36 hours, up to +2.8% day-7 retention, and triggered pushes reaching 12–21% CTR against a 3.7% average.
Want the receipts? Read the full story

The receipts

The full story: context, approach, examples and results.

Context

I took on email and push in late 2023, before we had a CRM platform. Campaigns were built and data pulled by hand, and every week I worked with a backend developer to get sends out. Nothing could recur on its own, and nobody could say whether a message changed behaviour or simply took credit for something users would have done anyway.

The challenge

Own lifecycle messaging end to end for around 2 million push and 16 million email contacts, make it move key actions, and prove it rigorously enough that the wider team would trust the numbers.

What I did

I pushed for a self-serve platform so comms no longer took up developer time. Our developers built the MoEngage integration, and from then on I ran email, push and in-app messaging directly, with support from the marketing team along the way.

I mapped the moments that mattered most, such as an unfinished claim, a first reward waiting, or a streak about to break, and built trigger flows around each one. Audiences were segmented by in-app behaviour, churn risk, language and, where relevant, country, with copy translated into every major language through MoEngage variations.

Important flows kept a random holdout group who received nothing, with the split sized to each campaign, and ran for anywhere from two weeks to over a month depending on how much rode on the result. Comparing the groups showed the true lift of each message. Alongside the holdouts, I ran A/B and A/B/C tests to find the strongest version of each message.

I watched unsubscribes closely during every test and never scaled a test send faster than the results justified. Anything that beat the holdout stayed, and anything that didn't was cut.

The top five pushes

Click-through rate over the last 90 days (5 July to 3 October 2026), for campaigns that reached at least 100K people. The average across all push was 3.7%.

  • 01

    Finish your claim · 21.1% CTR

    Sent when someone started a claim and didn't finish it. 802K received, 120K taps.

  • 02

    Finish your flip · 18.1% CTR

    Sent when someone left the Daily Flip game halfway through. 615K received, 78K taps.

  • 03

    Your first SWEAT · 17.6% CTR

    Sent to accounts under 14 days old when their first reward was ready. One line, "You earned your first SWEAT! Tap to claim", opening straight to the claim screen. 58% of people who tapped went on to claim.

  • 04

    Finish your spin · 16.0% CTR

    Sent when someone left Spin the Wheel before collecting. 371K received, 46K taps.

  • 05

    Keep your streak alive · 11.9% CTR

    Sent only to users with a live streak about to break, in eight languages. It held steady at 11.8–12.1% every week across 994K sends.

What made them work

All five fire on the user's own action, not a schedule: an unfinished claim, flip or spin, a first reward, or a streak at risk. Industry data shows the same pattern, with triggered pushes opening at 14.4% against 4.2% for broadcasts.

Each one lands on the screen where the job gets finished, not the home screen. The audiences are narrow and warm: our periodic pushes to the whole active base reached 1.4–4.2%.

Language mattered too. Translating the streak messages lifted their click-through by about 25%.

A tap isn't the same as impact

Click-through proves someone tapped, not that the message changed what they did. The reminders also fire within minutes of the user's last action, so some taps came from people who were still in the app. That's why I tested against holdouts. These are the pushes with control groups clean enough to trust, July to September 2026.

  • 01

    Your first SWEAT · +16%

    19.65% of people who got the push claimed within about 36 hours, against 16.90% held out. Day-7 retention rose by up to 2.8 points and day-30 by up to 1.4.

  • 02

    Spin the Wheel · +18%

    6.94% spun the same or next day, against 5.89% held out, across 2 million users.

  • 03

    Daily Flip · +21%

    5.94% flipped the same or next day, against 4.91% held out, across 2 million users.

  • 04

    Claim All · +25%

    2.74% claimed the same day, against 2.19% without the push (April to June 2026).

  • 05

    Account recovery · +49%

    A push asking people to set up a recovery key took setups from 0.31% to 0.46%. Small numbers, but a real security habit.

What I learned about email

Short wins. Everything important should be readable on open, without scrolling, with one obvious button where a thumb naturally rests at the bottom of a phone screen. Extra detail can sit below it, but never the essential or most engaging part.

Timing is worth testing. I tested send times, then let MoEngage schedule each send for the app's or each user's best time.

Over the last year our emails averaged a 24.1% unique open rate.

Results

Trigger flows lifted reward claims by 4.7% within 36 hours compared with the holdout, and improved day-7 retention by up to 2.8%. Email click-through reached 2.4%, against a 1.45% benchmark for the fitness industry.

The best triggered pushes reached 12–21% click-through against a 3.7% average, and every push with a clean holdout beat its control group, by 16% to 49%. Comms no longer needed a developer every week.

Don't put everything in the message. Let it tell its own story and point to the benefit, and let the app do the rest.
  • Email
  • Push
  • In-app
  • MoEngage
  • Data