Senior Product Manager - Experimentation
Sign up free to see how well your resume matches this role.
What you'll do
- Define and own the experimentation vision, strategy and improvement roadmap, balancing governance, measurement, platform and enablement priorities against business goals.
- Define and enforce the experimentation standards ASOS works to, reviewing upcoming experiments for quality and readiness before launch through a weekly review cycle.
- Own the experimentation measurement strategy, defining primary, secondary and guardrail metrics and driving improvement in statistical validity and conclusive rates.
- Drive adoption of advanced methodologies including Bayesian sequential testing, multivariant testing and multi-armed bandits, partnering with Analytics teams and external experts.
- Own Optimizely platform strategy, capability and health, and manage the vendor relationship including support, incident management and roadmap influence.
- Design AI-enabled experimentation workflows and agents, with governance for AI-assisted experimentation agreed before tooling deploys.
- Partner with Product Analytics and Data to improve experiment data quality, driving ADE integration, metric certification and pre-launch measurement readiness gates.
- Define the standard experimentation workflow, launch gates and rollout decision frameworks that catch quality issues before they become expensive.
- Run the Experimentation Forum and Champion network, and support Product Managers directly on experiment planning, training and best practice.
- Define and track the KPIs measuring experimentation capability health, and report programme performance to Product Directors and senior leadership.
- Work across Product, Engineering, Analytics, Trade and Data to keep experimentation aligned to company goals, communicating vision and outcomes clearly across the business.
- Champion quasi-experimental methods (e.g. diff-in-diff, synthetic control, matched market tests) for scenarios where randomised testing isn't feasible, and build a playbook to guide their consistent application.
What they're looking for
- Deep experience running experimentation or A/B testing programmes at scale across multiple teams, ideally in ecommerce or a consumer digital environment, with practitioner-level knowledge sufficient to credibly challenge a hypothesis, a metric definition or an experiment design.
- Hands-on expertise in frequentist and Bayesian approaches, sequential testing, multivariant testing and multi-armed bandits, sufficient to evaluate their suitability on merit and guide adoption.
- Expertise designing primary, secondary and guardrail metrics, and diagnosing why experiments return inconclusive results.
- Hands-on ownership of an enterprise experimentation platform such as Optimizely, including configuration, integration and capability assessment.
- Understanding of experiment data pipelines, event validation, metric certification and the causes of result inaccuracy.
- Experience defining standards, launch gates and quality processes that hold across teams you don't manage, with the ability to set and enforce them without direct authority.
- Experience applying product management to internal capability, tooling and process rather than customer-facing features, motivated by raising the capability of others.
- Awareness of how AI can be applied to analytical and operational workflows, and of the governance required when it is.
- Experience managing a strategic software vendor relationship, including commercial review and roadmap influence.
- Confident presenting to Director level, translating technical measurement issues into commercial consequences, and diagnosing recurring quality problems at root cause rather than case by case.
Nice to have
- A background in statistics, mathematics, economics or a related quantitative field is preferred, though equivalent practitioner experience will also be considered.
Summarised by NextRaise from the employer’s description, which follows in full below.
Full description from employer
Company Description
At ASOS, we're proud to be a global fashion destination serving over 23 million active customers across more than 200 markets with 2.5 billion visits annually. Our Tech team is at the heart of everything we do, powering the digital experiences that make ASOS a leader in online fashion retail. We operate as a product-led organisation, where cross-functional teams are empowered to solve real customer problems through innovation, experimentation, and data-driven decision-making. With a strong focus on scalability, experimentation, and cutting-edge technology, we're building the future of fashion commerce.
As the Experimentation Senior Product Manager, you'll be the focal point for how ASOS experiments, setting big, ambitious goals for the capability and taking ownership of how to meet them. You'll have a growth mindset and a can-do attitude and relish the autonomous culture. As a natural storyteller, you'll build solid narratives and take your stakeholders on the journey throughout.
Job Description
As the Experimentation Senior Product Manager at ASOS, you'll own the experimentation capability end to end the standards experiments are designed to, the metrics they are measured against, the platform they run on, the data feeding their results, and the governance determining whether a result is trusted enough to act on. This role does not run individual experiments; it determines whether hundreds of experiments run by other people produce decisions the business can rely on.
ASOS runs experimentation at scale across Product, Engineering, Analytics, Trade and Data, and the volume of experiments isn't the constraint, the capability surrounding them is. When that capability is strong, experiments produce clear answers quickly and the business acts on them with confidence; when it's weak, teams run experiments that can't conclude, measure the wrong things, or produce results nobody trusts. This role owns the difference between those two states.
You'll define the vision and roadmap for how ASOS experiments, and be accountable for the quality and reliability of experimentation outcomes across the business, setting the standards experiments are held to and enforcing them through a weekly review cycle, owning the measurement foundations that determine whether results can be trusted, owning the experimentation platform and its strategy, and building the capability of the wider organisation through the Experimentation Forum, the Champion network, training and direct support to Product Managers.
The role is roughly 55% operational leadership — governance, programme delivery, stakeholder management, enablement, leadership reporting and vendor management — and 45% technical product leadership — measurement frameworks, statistical methodology, platform ownership, data and analytics integration, and AI and automation. The 45% is the distinguishing feature: a candidate who can run the programme but cannot challenge a metric definition or assess a Bayesian approach will deliver roughly half of what the role requires.
You'll collaborate with Product Analytics, Analytics Engineering, Engineering leaders and Trade partners to build a trusted, high-quality experimentation capability that drives faster and more confident product decisions.
Key Responsibilities
Define and own the experimentation vision, strategy and improvement roadmap, balancing governance, measurement, platform and enablement priorities against business goals.
Define and enforce the experimentation standards ASOS works to, reviewing upcoming experiments for quality and readiness before launch through a weekly review cycle.
Own the experimentation measurement strategy, defining primary, secondary and guardrail metrics and driving improvement in statistical validity and conclusive rates.
Drive adoption of advanced methodologies including Bayesian sequential testing, multivariant testing and multi-armed bandits, partnering with Analytics teams and external experts.
Own Optimizely platform strategy, capability and health, and manage the vendor relationship including support, incident management and roadmap influence.
Design AI-enabled experimentation workflows and agents, with governance for AI-assisted experimentation agreed before tooling deploys.
Partner with Product Analytics and Data to improve experiment data quality, driving ADE integration, metric certification and pre-launch measurement readiness gates.
Define the standard experimentation workflow, launch gates and rollout decision frameworks that catch quality issues before they become expensive.
Run the Experimentation Forum and Champion network, and support Product Managers directly on experiment planning, training and best practice.
Define and track the KPIs measuring experimentation capability health, and report programme performance to Product Directors and senior leadership.
Work across Product, Engineering, Analytics, Trade and Data to keep experimentation aligned to company goals, communicating vision and outcomes clearly across the business.
Champion quasi-experimental methods (e.g. diff-in-diff, synthetic control, matched market tests) for scenarios where randomised testing isn't feasible, and build a playbook to guide their consistent application.
Develop frameworks to measure the financial impact of experiments, connecting experiment outcomes to commercial value and business KPIs.
Qualifications
About You
Deep experience running experimentation or A/B testing programmes at scale across multiple teams, ideally in ecommerce or a consumer digital environment, with practitioner-level knowledge sufficient to credibly challenge a hypothesis, a metric definition or an experiment design.
Hands-on expertise in frequentist and Bayesian approaches, sequential testing, multivariant testing and multi-armed bandits, sufficient to evaluate their suitability on merit and guide adoption.
Expertise designing primary, secondary and guardrail metrics, and diagnosing why experiments return inconclusive results.
Hands-on ownership of an enterprise experimentation platform such as Optimizely, including configuration, integration and capability assessment.
Understanding of experiment data pipelines, event validation, metric certification and the causes of result inaccuracy.
Experience defining standards, launch gates and quality processes that hold across teams you don't manage, with the ability to set and enforce them without direct authority.
Experience applying product management to internal capability, tooling and process rather than customer-facing features, motivated by raising the capability of others.
Awareness of how AI can be applied to analytical and operational workflows, and of the governance required when it is.
Experience managing a strategic software vendor relationship, including commercial review and roadmap influence.
Confident presenting to Director level, translating technical measurement issues into commercial consequences, and diagnosing recurring quality problems at root cause rather than case by case.
A background in statistics, mathematics, economics or a related quantitative field is preferred, though equivalent practitioner experience will also be considered.
Additional Information
Benefits
- Employee discount (hello ASOS discount!)
- Employee sample sales
- 25 days paid annual leave + an extra celebration day for a special moment
- Discretionary bonus scheme
- Private medical care scheme
- Flexible benefits allowance - which you can choose to take as extra cash, or use towards other benefits
- Opportunity for personalised learning and in-the-moment experiences that enable you to thrive and excel in your role
Company
Company facts come from this company's own listings. We only show what the postings themselves carry.