AI's Landslide Win: Generated Pushes Beat Humans, Deliver +68% Conversions for Promova

Intro

In a recent case study, we showed how Promova, a top language learning platform trusted by 17 million users worldwide, boosted message CTR and DAU by 10% with automated lesson reminders.

This case study explores how Reteno’s “One from many” messaging block helped Promova achieve a 82.3% higher click-through rate (CTR) and a 68.1% higher conversion rate (CVR) compared to the control group by switching from manual messaging to AI-generated push notifications. These automated triggers adapt to real user behavior. They outperformed human-crafted content, boosting engagement and conversions.

Challenge

Promova’s marketing team faced a major growth challenge: maintaining personalized engagement across their multi-million user base while scaling globally. Their prior success with automated lesson reminders proved the power of timely communication throughout the user journey. However, manually writing high-performing notifications was time-consuming and inconsistent.

The key question was: could AI-generated messages, based on data-driven insights and optimized language, outperform human-written notifications in click-through rates (CTR) and conversions?

The hypothesis was that AI-generated triggers, personalized to user behavior, would yield better engagement metrics than manually-crafted nudges. However, this required a scalable, automated solution that could test and refine notification variants without disrupting Promova’s existing communication flow.

Solution

Promova’s marketing team partnered with Reteno, a marketing support platform. Reteno’s experts used the “One from many” messaging block and set up a dynamic A/B test. The test compared the effectiveness of manually written push notifications against AI-generated options. Since each message was part of a single, unified workflow, there was no need to build separate branches or duplicate flows.

This experiment marked the first use of Reteno’s built-in AI for creating mobile push notifications. As part of the “One from many” solution, this tool automatically generates localized and engagement-optimized message content by combining behavioral data, user context, and performance history.

The system then delivered AI-generated nudges, enabling the flow to evaluate them in real time and automatically scale the ones with the highest engagement metrics.

The testing process unfolded as follows:

1. Message creation. Reteno crafted push notifications using data-driven insights and tailoring content to users’ preferences and behavioral patterns. Messages like “Pick up your first phrases today” were designed to be clear, motivating, and benefit-focused. They contrasted with manually written nudges, which often lacked precision.
2. Automated A/B testing. The “One from many” block sent both message variants to specific user segments, measuring CTR and conversions. This allowed marketers to directly compare AI-generated content performance against human-written variations.
3. Real-time optimization. Reteno automatically identified top-performing push messages and scaled them to broader audiences without manual oversight. As demonstrated by the raw data, this resulted in content delivery that was exceedingly effective in generating conversions.

Results

The A/B test results confirmed the hypothesis: AI-generated messages consistently outperformed their human-written counterparts, driving notable improvements in engagement metrics.

Key Findings

• The top-performing AI-generated mobile push, “First lesson unlocked!” delivered the strongest results. Compared to the control group, there was an 82.3% increase in the click-through rate (CTR) and a 68.1% increase in the conversion rate (CVR) during the test.

• By contrast, pushes like “Your language journey begins!” had a positive tone but were less specific about the benefits. Compared to the top performers, this resulted in a lower (58.8%) deviation in test/control group conversions.

• Generic nudges like “Begin your language adventure!” had the weakest results. Lacking specific incentives or urgency, these messages failed to effectively engage users (-33.7% CVR compared to the control group).

The experiment also highlighted the importance of monitoring performance variability and continuously optimizing AI prompts and data inputs. While not all automated messages perform equally well, tracking real-time engagement metrics allows marketers to swiftly identify and maximize the impact of the most successful alerts.

Why This Matters for You

Promova’s success with AI-generated messaging provides actionable insights for any app seeking to boost engagement and conversions. Here’s what you can learn from this case study:

✅ AI consistently outperforms manual efforts. Messaging automation helps you create personalized, data-driven content that elevates your app’s CTR and conversions beyond what human writing achieves.

✅ Automated A/B testing saves time and resources. Reteno’s “One from many” block automates message testing and refinement throughout the user journey. The result? With minimal effort, you can significantly enhance your value proposition.

✅ Clarity and specificity drive action. Communicate messages with specific benefits and urgent calls to action (e.g., “Start now!”). They engage better than vague or abstract ones.

✅ Personalization boosts conversion. Engage diverse audiences by delivering AI-tailored content based on user preferences and app usage history.

✅ Scalability made simple. Scale user engagement with automated 1:1 communications to keep your app top of mind as your contact base grows.

Ready to boost your app’s conversions and refine the user experience like Promova? Adopt Reteno’s AI-driven messaging and book a demo today!