Read out an A/B test result honestly
Use to interpret an experiment without overclaiming significance or ignoring practical effect size.
Act as an experimentation statistician who refuses to overclaim.
Experiment: {{experiment_name}}
Metric: {{primary_metric}}
Control: {{control_n}} users, {{control_value}}
Variant: {{variant_n}} users, {{variant_value}}
Do the following:
1. Compute the observed lift and its 95 percent confidence interval.
2. Run the appropriate test (proportion z-test or t-test) and report the p-value.
3. State whether the result is statistically significant AND whether the effect is practically meaningful.
4. Check for obvious traps: peeking, sample ratio mismatch, underpowered design.
5. Give a one-paragraph plain-language recommendation: ship, kill, or keep running.Click the copy button in the top right of the block to grab the full prompt.
Replace each placeholder below with your own values before you run the prompt.
- {{experiment_name}}
- {{primary_metric}}
- {{control_n}}
- {{control_value}}
- {{variant_n}}
- {{variant_value}}
Related prompts
You are a meticulous data analyst. Here is a sample of my raw dataset (header row plus a few rows): {{sample_rows}} Context: this data describes {{data_description}} and I plan to...
You are a senior Python data engineer. Write clean, reproducible pandas code to clean my dataset. Columns and their meaning: {{column_spec}} Known issues to handle: {{known_issues}...
Act as a data analyst guiding me through exploratory data analysis. Dataset summary: {{dataset_summary}} My goal / the question I care about: {{analysis_goal}} Produce an EDA plan...
You are a SQL expert writing for a {{sql_dialect}} database. Here is the relevant schema (tables, columns, types, keys): {{schema}} Question to answer: {{question}} Rules: - Use on...
You are a SQL reviewer. Explain and debug the query below. Dialect: {{sql_dialect}} What I expected it to return: {{expected_result}} What is actually wrong (if known): {{symptom}}...
You are a database performance specialist for {{sql_dialect}}. Slow query: {{query}} Context: - Approximate row counts of the main tables: {{table_sizes}} - Existing indexes: {{ind...
0 Comments
Loading discussion...