Skip to content

Latest commit

 

History

History
32 lines (23 loc) · 1.65 KB

README.md

File metadata and controls

32 lines (23 loc) · 1.65 KB
Raw

Mobile Games: A/B Testing

A/B test is to be analyzed from the popular mobile puzzle game, Cookie Cats. The data is about A/B test with a mobile game, Cookie Cats. The data link is written here , please have a look.

The purpose of this repo is to show my understandings on A/B test. It is one practical experience for A/B Test.

In the dataset,

  • two version/groups: gate_30 & gate_40,
  • measureable result: retention_1, retention_7 & sum_gamerounds

A/B Testing Procedure

  1. Understand the Problem

    1. If mobile game shall adjust its version from gate_30 to gate_40 so that the game rounds could increase and 1-day retention and 7-day retention could decrease.

  2. Hypothesis Testing

    1. Set Up Hypothesis

      1. Null Hypothesis (H0): There is no statistical difference between gate_30 && gate_40
      2. Alternative Hypothesis (Ha) : There is statistical difference between two groups.

    2. Analysis the result i) Because the data is given, choose the statistical method

      1. method 1:Mann Whitney U Test Result
      2. method 2: Bootstrapping multiple times and find difference two versions'median value. Generate the density plot to show

  3. Interpret results If the result is lower than threshold (p-value), then H0 hypothesis will be rejected, in other word, there is a statistically signficiant difference between them. Else, there is no statistical difference between them.

Test Result

Conclusion

Becuase there is no significant difference, thus, there is no need to move from gate_30 to gate_40 by AB Test.