Case study

ZTE — App Discovery & Monetization

Making an app store's recommendations pay for themselves — AI-driven discovery and monetization at platform scale.

  • Platform ML
  • Monetization
  • Game Folder
  • Android UI / Axon
  • A/B testing
  • Developer ecosystem
Role
Director of Product
Timeframe
2014 – 2018
Platform
Android platform · global
Status
Shipped
TL;DR
  • Owned a software-monetization program that used ML-driven recommendations to improve app discovery on ZTE's Android platform, with conversion and revenue growth as the only roadmap criteria.
  • Shipped the Game Folder — recommendations surfaced inside the home-screen games folder, at the moment of intent — the placement that proved format and context beat model quality; partnered with data science and ML engineering on model selection and training data and ran A/B tests on format and placement.
  • Built the performance-monitoring and senior-stakeholder reporting that turned a recommendation engine into a line item leadership could manage; separately ran global developer-marketing and hackathon programs for a dual-screen platform launch with AT&T and Cisco.

Context & problem

An OEM’s software revenue lives in the margins: preloads, store placements, recommendation slots. ZTE shipped tens of millions of Android devices and had a store surface on each one, but discovery was a static shelf — editorial placements, no personalisation, low conversion, and monetization that depended on who negotiated the preload deal.

The mandate was to turn that surface into a revenue program: use recommendations to lift discovery, and prove it in numbers senior leadership would fund.

Insight

A recommendation engine is a product only once it has a P&L. The data-science team could build a better model; what was missing was a definition of “better” that tied to money. Conversion-per-impression and revenue growth were set as the success metrics before model work began, and every subsequent decision — model choice, training data, ad format, placement — was evaluated against them. That sounds obvious and is routinely skipped; it is the reason this program survived three budget cycles.

The second insight was about what to test. Model quality moved the numbers less than format and placement did. The cheapest A/B tests — where the recommendation appeared, how it was labelled, how many slots — produced the largest monetization gains.

Mini PRD

ProblemStatic app-store surfaces under-monetize a large installed base; discovery is editorial, not personalised.
UsersDevice owners (discovery) · App developers and ad partners (demand) · ZTE software revenue (owner).
GoalsLift conversion per impression · Grow software revenue per device · Make recommendation performance legible to leadership.
Non-goalsRebuilding the store client · Competing with Google Play on catalogue.
Core requirementsML-driven recommendation engine with defined training-data pipeline · Experimentation framework for ad format and placement · Performance monitoring with conversion and revenue dashboards · Senior-stakeholder reporting cadence.
Success metricsConversion per impression · Revenue growth (software line) · Engagement with recommended content.
ConstraintsPrivacy and regional data rules across markets · Carrier placement agreements · On-device performance budgets.
Open questionsHow far personalisation could go within carrier and regional constraints.

Strategy & bets

  • Metrics first, model second. Conversion and revenue were the roadmap; model improvements were funded only when they moved those.
  • Test the cheap levers hardest. Format and placement experiments ran continuously; model experiments were batched.
  • Make it reportable. Standing performance monitoring and a senior-stakeholder reporting rhythm turned the engine from a project into a managed revenue line.
  • Ecosystem as a parallel bet. For a dual-screen platform launch, I designed and ran global developer-marketing and hackathon programs with AT&T and Cisco across multiple regions — recommendations are only as good as the catalogue behind them.

The Game Folder: recommendations where the user already is

The clearest product expression of the monetization program was the Game Folder. Every Android home screen has a folder where the user drops their games; on ZTE devices that folder became a surface. Open it and, below the installed games, the recommendation engine fills a row with games the user is likely to want next — surfaced in the folder itself, at the moment the user has just signalled they’re in a gaming mood, rather than in a store they’d have to go and open.

That placement decision did more for the numbers than any model change. A recommendation in a store competes with everything else in the store; a recommendation in the game folder is the only thing on screen that isn’t already installed, shown to a user who is about to play. Conversion per impression on the folder surface was the benchmark every other placement was measured against, and it’s the example I use when I say format and placement beat model quality: the engine was the same, the context was different, and the context was the product.

The Game Folder is also where the developer-ecosystem work paid off directly — the folder was only as good as the catalogue of games with partnership terms behind it, so the recommendation slot, the developer program and the revenue share were designed as one product, not three. Demand came through three partners: Google — via its ad exchange product, which put the folder’s inventory in front of programmatic buyers rather than a single network — plus InMobi and ironSource as direct game-install demand, each filling the slot on different terms, with the engine deciding which partner’s inventory a given user saw. Managing three demand sources against one surface — yield per partner, fill rate, and the user-experience floor below which no payout was worth it — was its own small ad-ops product inside the program.

Two UIs, two markets: Nubia UI and the US Axon UI

Alongside the monetization program, I owned the product definition of ZTE’s Android experience for the US — and that meant deciding what to do with a UI that had been built for a different market.

Nubia UI was ZTE’s China-market skin: feature-rich, heavily customised, built around a domestic ecosystem of services and a user base that expected the OEM to replace most of Google’s surface. Axon was the US flagship line, sold through and certified by US carriers, to users who expected Android to look like Android and Google services to be first-class.

Shipping Nubia UI to the US would have been cheapest and wrong. The US Axon UI was defined as a deliberately light layer: near-stock Android with Google services intact, ZTE differentiation concentrated in a few places that carriers and reviewers would actually notice (camera, audio, the dual-screen experience on Axon M) rather than spread across every system screen. The hard part was organisational — making the case inside a China-headquartered company that less customisation was the US product, and holding that line through every release where a feature from the China roadmap wanted a home on the US device.

Three rules came out of it:

  • Differentiate where the market looks. Reviewers and carriers judged camera, audio, display and the dual-screen use cases; nobody rewarded a custom dialer.
  • Carrier certification is the schedule. Every divergence from stock was a certification risk and a delay; the lighter the layer, the faster the launch.
  • One platform, two expressions. Nubia UI and the Axon UI shared the underlying platform work (performance, security patches, Google compatibility); only the experience layer forked. That kept two markets from becoming two engineering organisations — the same instinct as the single-SKU program at Sprint.

System design

Device signalsusage · installs · region Training datapipeline + labelling Rec. enginemodel selection w/ DS Experimentsformat · placement · slots Store surfaceimpressions conversion · revenue → monitoring → stakeholder reporting → next experiment Success metricsconv/impression · revenue
Fig. 1 — The metrics box is upstream of both the model and the experiments. Everything reports into it.

I was not the ML engineer here and don’t claim to be; the product work was choosing which model and which training data with the data-science team, deciding what to experiment on, and building the instrumentation so the results meant something to a VP.

Metrics & experimentation

  • Success metrics: conversion per impression and revenue growth, fixed at program start.
  • Continuous A/B testing on ad format, placement and slot count; model changes evaluated on the same two numbers.
  • Standing performance monitoring and a senior-stakeholder reporting cadence — the same dashboard served the data-science team and the business review.

Outcomes

  • A software-monetization program with ML-driven recommendations running at platform scale, with a defined roadmap for iterating the engine on conversion and revenue.
  • Experiment program on format and placement that drove engagement and monetization gains.
  • Developer-ecosystem programs delivered in partnership with AT&T and Cisco across multiple regions for a dual-screen platform launch.

Retro — what I’d do differently

  • Instrument the counterfactual. We measured lift against the previous period more than against a true holdout; a permanent control cell would have made the revenue attribution unassailable.
  • Push harder on the catalogue. Recommendation quality was ultimately bounded by what was in the store; I’d have started the developer-ecosystem work a year earlier and tied it to the same metrics.