× Patterns Blog BetterUI Review Coaching Courses Datastories

Running Your First A/B Test – A Draft Course Outline

We have moved forward with a draft outline for the optimization course we’re preparing and we’d love to hear your feedback. Last week some of you already shared some questions to steer us and then we also validated the need for such the course itself. Thanks for that. This week, here is another update.

Draft Course Title: Running Your First A/B Test

And we already have a really cool company which we’ll work with to optimize their site as the core subject. :)

The Tools We’ll Use In The Course

Some of the tools we’re thinking of using in our course include: VWO for A/B testing, Evan Miller’s Sample Size Calculator, Thumbtack’s ABBA Split Test Calculator, Google Docs for idea prioritization, GoodUI Evidence for testing inspiration, and Trello for test management. That’s at least what we have been using for the last few years while a/b testing.

The Course Outline

Please Tell Us Your Thoughts About The Outline

What do you think about the above outline? Do you wish for us to focus on anything else during the course? Now is the time to let us know. (The “days” are just used for reference. We might decide to refer to actual days from the real project.)

Share this story

Posted by Jakub Linowski on Mar 24, 2016

Leave Your Reply

11 Comment threads
3 Thread replies
Most reacted comment
Hottest comment thread
12 Comment authors
Carolina Avila CoralJennifer McKayThomas TonderRuggeroJakub Linowski Recent comment authors
newest oldest most voted

This looks really comprehensive! All your resources are amazing so seeing behind curtains is really exciting!!

I think you should include something about using heatnaps in the indentifying Testing Opportunities section.

Also it would be interesting in the extract finding section to use heatmaps to compare the difference in clicks on each test variation.

Stacy Sutton Williams

I would love to have my team take this. How much will it cost?

Jakub Linowski
Jakub Linowski

Hi Stacy. We still haven’t decided on the pricing. It might be offered as part of Datastories, is one idea.

Dominick Branigan

I think this is going to be great and I look forward to the course. I would love to see specific A/B testing such as, visitor-to-lead conversion rate, maybe landing page, email, or call-to-action A/B test. As I very new to A/B testing it would also be cool to see commonly tested elements

Robert Brown

When will you do this and how much will it cost? Will it be a webinar course? We actually have a web application product.


Looks amazing Jakub! I really hope you consider LifterLMS when you go to build it :-) I’d love to make a custom theme for you :-)

Michael Facchinello
Michael Facchinello

I am confused how it jumps from day 8 to day 30. A bit of clarification around that could be good.

David Frahm

I would want this to touch on how to know when to stop. Might be a bit in the Setting Focus as far as what type of gains you are looking for, and then in the post analysis how to know whether to go again/deeper on the same test or move on. Basically discussing incremental vs exponential gains.

Jakub Linowski

Agreed, that this could be interesting. We are also actively researching this and have tried to stir a discussion on stopping rules right here: http://goodui.org/blog/agile-ab-testing-using-stop-rules-to-minimize-losses-time-wasted/ Care to share your stopping rules if you have any?


Great plan!
I’d like to deep the tools sections: how to use them and how to correctly read the results of the tests.

Thomas Tonder
Thomas Tonder

Looks really great !
Would love to see something about the tools you use to work as a team during the process (ideas backlog, bugs/correction tracking, global project management, reporting of results…)
Thanks for sharing with so much passion :)

Jennifer McKay
Jennifer McKay

Jakub, I’m really impressed with what you’re doing here (with goodui). This class structure sounds really strong and I’m very interested in taking this. I really appreciate the community here.

Carolina Avila Coral

Hi Jakub:

I think it will be an amazing course.
I’m not native English speaker, so, I hope you save the class in a video so I can watch the class many times until understand, in addition, I will appreciate a lot if the videos could have subtitles or a closed caption.

Thanks a lot.