A/B testing using a real marketing funnel dataset, plus a synthetic dataset I built myself to check whether my own testing method actually works.

Read implementation of A/B testing on a bigger, real data set here! [LINK]

Big-picture method

• Loaded the test and control files separately, tagged each one so I could tell them apart, then combined into one table.
• Checked for missing data. Found one row that was basically empty, so I dropped it instead of letting it mess up totals later.
• Turned the raw counts into daily rates. Click-through rate, cart rate, purchase rate, calculated per day.
• Made a few different charts of the funnel. Total counts on a log scale, stage-to-stage conversion percentage, a normalized retention curve, a time series, and a triangle-shaped funnel chart. Summarised my findings as well as the pros and cons of each graph.
• Checked whether the data was normal before picking a test instead of just defaulting to a t-test.
• Ran a t-test and a Wilcoxon test at every stage, side by side, so I could see if they agreed.
• Checked cost efficiency

Loading and cleaning the data

The data comes as two separate files, not one combined file, so the first step was loading both, tagging each with which group it belongs to, and stacking them into one table. Then, cleaning them to make sure there are no incomplete rows.

test_raw <- read_delim("test_group.csv", show_col_types = FALSE)
control_raw <- read_delim("control_group.csv", show_col_types = FALSE)
test_raw$campaign <- "test"
control_raw$campaign <- "control"
df <- bind_rows(test_raw, control_raw)

One row in the control group was missing almost everything, just a date and a spend amount. I dropped it instead of letting it turn every downstream calculation into NA

This is the information available to us for both the groups:

• Date
• Number of observations (30)