It’s one thing to understand the difference between Type 1 and Type 2 errors. And another to remember the difference between Type 1 and Type 2 errors! If the man who put a rocket in space finds this challenging, how do you expect students to find this easy!?
If you teach statistics, you already know this! Students from all walks of life, MBAs, medicine, arts, STEM, etc., are troubled when asked to differentiate between Type 1 and Type 2 errors! And don’t get me started on type 3 or type 4 errors! Here are Tamsin and Caitlin on Twitter ranting about Type 1 and Type 2 errors!
Tamsin, @anorakgirl, says “Statistics people! How am I supposed to remember anything for my exam when your naming is SO bad? Type 1 and Type 2 errors are just the last straw…”
Caitlin, @Saiki_36, “I’ve read 2 chapters and watched the videos on type 1 and 2 errors for stats and retained nothing. Goodbye brain.”
In fact, @stianwestlake, CEO of nonother than “Royal Statistical Society,” concludes.
To help, we address on this page:
- What are Type 1 and Type 2 errors?
- Seven ways to remember Type 1 and Type 2 errors.
- Real life examples of badly mixing up Type 1 and Type 2 errors.
- Trade-off between Type 1 and Type 2 errors.
What are Type 1 and Type 2 Errors?
Before we try and remember Type 1 and Type 2 errors, it is good first to understand the concept.
In statistics (and in life), we often start with a null hypothesis (a belief we hold) and test it using data (or evidence). Then, we evaluate the sample data (or evidence gathered) to conclude if we continue with the null hypothesis (or change belief we hold). It is always possible we make the wrong conclusion because the data is a sample only, or evidence may be insufficient. There are two ways of making this error. The two ways were named Type 1 error and Type 2 error.
- A type I error occurs when we reject a null hypothesis that is actually true in the population.
- This is also referred to as a false-positive.
- Measured by Alpha.
- A type II error is when we fail to reject a null hypothesis that is actually false in the population.
- This is also referred to as false-negative.
- Measured by Beta.
Students usually ask why we do not drop the type 1 and type 2 error terminology and call it a false positive and false negative! Technically, we use the type 1 and type 2 error terminology for hypothesis testing in statistics. Clearly, there could have been better ways to name these errors!
If you are looking for a primer on hypothesis testing, the Industrial Psychiatry Journal has one here.
Easy Ways to Remember Type 1 and Type 2 Errors
We have searched the internet and submit different ways that people have tried to remember the difference between Type 1 and Type 2 errors.
1) The Wolf Story to remember the difference between Type 1 and Type 2 errors
This is the most popular method to remember the difference between Type 1 and Type 2 errors.
Everyone probably knows the story of the boy who cried wolf! You only need to remember that the villagers committed both Type 1 and Type 2 errors in that order.
- When the boy cries wolf the 1st time, the villagers believe it to be true and rush to the scene but there’s no wolf. This is a false positive or Type 1 error.
- Then the boy cries wolf the 2nd time when a wolf is encountered and the villagers ignore or don’t believe there wolf. This is a false negative or Type 2 error.
2) Pregnancy & the difference between Type 1 and Type 2 errors
The close second is the pregnancy image doing the rounds on the internet.
3) The Rain and Umbrella & the difference between Type 1 and Type 2 errors
4) The DIY Calligrapher & the difference between Type 1 and Type 2 errors
If you are a calligrapher and want to make your own image -something like this is good.
5) Creating Your Own Memory Aid
@manishaaaaxo tried the approach here!
- Type 1…
- another word for 1 is First
- First begins with the letter F
- F = false (actual) hypothesis which was accepted Then you know that for the Null Hypothesis it’s just the opposite i.e. it was true but it was rejected
- Therefore, you just need to remember “First False” and remember that the false is referring to the ACTUAL hypothesis
- Type 2… think “too true” = 2 true The (actual) hypothesis is TRUE but it was rejected. Again the Null would just be the opposite- it was false but accepted
- So if you remember either First False or Too True you’ll be fine bc you just know that it’d be the opposite
6) Number of Falses’ Method
Here is another memory aid from Professor Kelsey-Jo.
7) The Tradition Matrix Showing Type 1 and Type 2 Errors
And of course, you have the traditional method to understand and remember the difference between Type 1 and Type 2 errors in Hypothesis testing. Sometimes, the traditional way works best!
@MoeMKiss and @MarqJacobs prefer this matrix. Note the change in the x-axis and y-axis.
You now have at least seven ways to remember and distinguish between Type 1 and Type 2 errors. Pick whatever suits you best
Examples of Misunderstanding Type 1 and Type 2 Errors
Elon Musk’s response to his COVID-19 Tests
Cant Call Type 1 & 2 errors “positively toward negative, right”
Unless you are a sitting president. Watch Trump’s Response on His COVID-19 Tests
Trump on his latest coronavirus test: “I tested very positively in another sense, so, this morning, yeah. I tested positively toward negative, right? So, no. I tested perfectly this morning, meaning I tested negative. But that’s a way of saying it. Positively toward the negative”
On Fox News
Trump on his latest coronavirus test: "I tested very positively in another sense, so, this morning, yeah. I tested positively toward negative, right? So, no. I tested perfectly this morning, meaning I tested negative. But that's a way of saying it. Positively toward the negative" pic.twitter.com/El53NhCqOL
— Aaron Rupar (@atrupar) May 21, 2020
Another Trump response on his colleague’s COVID-19 test results.
Trump: Katie, she tested very good for a long period of time and then all of the sudden today she tested positive… This is why the whole concept of tests aren’t necessarily great
Trump: Katie, she tested very good for a long period of time and then all of the sudden today she tested positive… This is why the whole concept of tests aren’t necessarily great pic.twitter.com/iM9Xo3jddU
— Acyn (@Acyn) May 8, 2020
Covid-19 Response
It is not just Donald Trump; the White House response was also unclear on potential Type I and Type II errors!
@MattGertz, Senior Fellow at @mmfa (and one who takes care to clarify that he is definitely not the GOP congressman) said on Twitter, “Dawning realization that nine months into the pandemic. senior WH officials think that if you test negative, it means that you definitely have not caught the virus and can do whatever you want and don’t need to quarantine if you’ve been in contact with someone with COVID.”
Trade off between Type 1 and Type 2 errors
Remember that there is always a trade-off between Type 1 and Type 2 errors. When you try and reduce Type 1 errors, you increase the odds of Type 2 errors. Therefore, controlling for Type 1 and Type 2 errors is based on the situation you are in.
@ProfJessieSmith points to The Texas Tribune article on bail reform to illustrate the trade-off between type 1 and type 2 errors.
Another good example for to show that controlling for Type 1 vs. Type 2 errors is based on the situation you are in is the example of screws. Assume you want to set the quality cut-off for boxes of screws. If the screws are used in airplanes, you want to minimize Type 2 errors as the cost of an error is an airplane failure – lethal and expensive. On the other hand, if the screws are used in a wooden table, you want to minimize Type 1 errors as the cost of an error is minimal.
PS: Please run if anyone mentions type 3 or type 4 errors! Don’t even reply!
Enjoy learning! Let us know if we can help you with tutoring, and one of our statistics tutors will be glad to assist you.