Daniel Burstein

Basic Validity Tool Download

July 12th, 2012

Thanks for registering for Bad Data: The 3 validity threats that make your tests look conclusive (when they are deeply flawed).

Date: Wednesday, September 14, 2011
Time: 4:00 PM – 5:00 PM EDT (US)

Educational funding provided by HubSpot.

To download your free basic validity tool, simply click the link below:

Download the free validity tool >>

  1. September 12th, 2011 at 15:03 | #1

    Not able to download validity tool.

  2. September 12th, 2011 at 19:35 | #2

    Thanks for the validity tool download. I’m running an experiment at the moment where Google Website Optimizer gives my treatment a 76.5% chance of beating the control, but your spreadsheet says that the results have achieved a Level of Confidence over 95% and the test is therefore conclusive. Can you please explain this apparent anomaly?

  3. September 13th, 2011 at 09:58 | #3

    Hi @J,

    Sorry about that. It should be fixed now. Let me know if you have any more problems.

  4. September 13th, 2011 at 10:56 | #4

    Stewart, good observation, I’m glad that you are double checking the results across multiple tools. Our attempt with this basic tool was to make it simple and easy to use. This meant that we had to make some assumptions and approximations. For example, we are only using aggregate data so we have made the assumption that the data being entered into the tool is normally distributed. We also are using an approximation for the standard deviation rather than calculating a standard deviation based on multiple data points. If these two assumptions are violated then the level of confidence returned by this basic tool will be higher than the level of confidence returned by a tool which has access to the raw data, such as GWO, or the full test protocol that we use internally.

  5. September 13th, 2011 at 18:58 | #5

    Thanks for the reply, Phillip. That makes sense.

  6. Jennifer Smith
    January 18th, 2013 at 10:41 | #6

    Hi – thank you for this – although when I download I unzip several xml files and don’t see the main workbook file…

  1. No trackbacks yet.