or “A lesson I wouldn’t have learned without technology”
Last November, some of my AP Statistics students were solving a problem involving a normal distribution with an unknown mean. Leveraging the TI Nspire CAS calculators we use for all computations, they crafted a logical command that should have worked. Their unexpected result initially left us scratching heads. After some conversations with the great folks at TI, we realized that what at first seemed perfectly reasonable for a single answer, in fact had two solutions. And it took until the end of this week for another student to finally identify and resolve the mysterious results. This ‘blog post recounts our journey from a questionable normal probability result to a rich approach to confidence intervals.
THE INITIAL PROBLEM
I had assigned an AP Statistics free response question about a manufacturing process that could be manipulated to control the mean distance its golf balls would travel. We were told that the process created balls with a normally distributed distance of 288 yards and a standard deviation of 2.8 yards. The first part asked students to find the probability of balls traveling more than an allowable 291.2 yards. This was straightforward. Find the area under a normal curve with a mean of 288 and a standard deviation of 2.8 from 291.2 to infinity. The Nspire (CAS and non-CAS) syntax for this is:
[Post publishing note: See Dennis’ comment below for a small correction for the non-CAS Nspires. I forgot that those machines don’t accept “infinity” as a bound.]
As 12.7% of the golf balls traveling too far is obviously an unacceptably high percentage, the next part asked for the mean distance needed so only 99% of the balls traveled allowable distances. That’s when things got interesting.
A “LOGICAL” RESPONSE RESULTS IN A MYSTERY
Their initial thought was that even though they didn’t know the mean, they now knew the output of their normCdf command. Since the balls couldn’t travel a negative distance and zero was many standard deviations from the unknown mean, the following equation with x representing the unknown mean should define the scenario nicely.
Because this was an equation with a single unknown, we could now use our CAS calculators to solve for the missing parameter.
Something was wrong. How could the mean distance possibly be just 6.5 yards? The Nspires are great, reliable machines. What happened?
I had encountered something like this before with unexpected answers when a solve command was applied to a Normal cdf with dual finite bounds . While it didn’t seem logical to me why this should make a difference, I asked them to try an infinite lower bound and also to try computing the area on the other side of 291.2. Both of these provided the expected solution.
The caution symbol on the last line should have been a warning, but I honestly didn’t see it at the time. I was happy to see the expected solution, but quite frustrated that infinite bounds seemed to be required. Beyond three standard deviations from the mean of any normal distribution, almost no area exists, so how could extending the lower bound from 0 to negative infinity make any difference in the solution when 0 was already standard deviations away from 291.2? I couldn’t make sense of it.
My initial assumption was that something was wrong with the programming in the Nspire, so I emailed some colleagues I knew within CAS development at TI.
GRAPHS REVEAL A HIDDEN SOLUTION
They reminded me that statistical computations in the Nspire CAS were resolved through numeric algorithms–an understandable approach given the algebraic definition of the normal and other probability distribution functions. The downside to this is that numeric solvers may not pick up on (or are incapable of finding) difficult to locate or multiple solutions. Their suggestion was to employ a graph whenever we got stuck. This, too, made sense because graphing a function forced the machine to evaluate multiple values of the unknown variable over a predefined domain.
It was also a good reminder for my students that a solution to any algebraic equation can be thought of as the first substitution solution step for a system of equations. Going back to the initially troublesome input, I rewrote normCdf(0,291.2,x,2.8)=0.99 as the system
and “the point” of intersection of that system would be the solution we sought. Notice my emphasis indicating my still lingering assumptions about the problem. Graphing both equations shone a clear light on what was my persistent misunderstanding.
I was stunned to see two intersection solutions on the screen. Asking the Nspire for the points of intersection revealed BOTH ANSWERS my students and I had found earlier.
If both solutions were correct, then there really were two different normal pdfs that could solve the finite bounded problem. Graphing these two pdfs finally explained what was happening.
By equating the normCdf result to 0.99 with FINITE bounds, I never specified on which end the additional 0.01 existed–left or right. This graph showed the 0.01 could have been at either end, one with a mean near the expected 284 yards and the other with a mean near the unexpected 6.5 yards. The graph below shows both normal curves with the 6.5 solution having an the additional 0.01 on the left and the 284 solution with the 0.01 on the right.
The CAS wasn’t wrong in the beginning. I was. And as has happened several times before, the machine didn’t rely on the same sometimes errant assumptions I did. My students had made a very reasonable assumption that the area under the normal pdf for the golf balls should start only 0 (no negative distances) and inadvertently stumbled into a much richer problem.
A TEMPORARY FIX
The reason the infinity-bounded solutions didn’t give the unexpected second solution is that it is impossible to have the unspecified extra 0.01 area to the left of an infinite lower or upper bound.
To avoid unexpected multiple solutions, I resolved to tell my students to use infinite bounds whenever solving for an unknown parameter. It was a little dissatisfying to not be able to use my students’ “intuitive” lower bound of 0 for this problem, but at least they wouldn’t have to deal with unexpected, counterintuitive results.
Surprisingly, the permanent solution arrived weeks later when another student shared his fix for a similar problem when computing confidence interval bounds.
A PERMANENT FIX FROM AN UNEXPECTED SOURCE
I really don’t like the way almost all statistics textbooks provide complicated formulas for computing confidence intervals using standardized z- and t-distribution critical scores. Ultimately a 95% confidence interval is nothing more than the bounds of the middle 95% of a probability distribution whose mean and standard deviation are defined by a sample from the overall population. Where the problem above solved for an unknown mean, on a CAS, computing a confidence interval follows essentially the same reasoning to determine missing endpoints.
My theme in every math class I teach is to memorize as little as you can, and use what you know as widely as possible. Applying this to AP Statistics, I never reveal the existence of confidence interval commands on calculators until we’re 1-2 weeks past their initial introduction. This allows me to develop a solid understanding of confidence intervals using a variation on calculator commands they already know.
For example, assume you need a 95% confidence interval of the percentage of votes Bernie Sanders is likely to receive in Monday’s Iowa Caucus. The CNN-ORC poll released January 21 showed Sanders leading Clinton 51% to 43% among 280 likely Democratic caucus-goers. (Read the article for a glimpse at the much more complicated reality behind this statistic.) In this sample, the proportion supporting Sanders is approximately normally distributed with a sample p=0.51 and sample standard deviation of p of . The 95% confidence interval is the defined by the bounds containing the middle 95% of the data of this normal distribution.
Using the earlier lesson, one student suggested finding the bounds on his CAS by focusing on the tails.
giving a confidence interval of (0.45, 0.57) for Sanders for Monday’s caucus, according to the method of the CNN-ORC poll from mid-January. Using a CAS keeps my students focused on what a confidence interval actually means without burying them in the underlying computations.
That’s nice, but what if you needed a confidence interval for a sample mean? Unfortunately, the t-distribution on the Nspire is completely standardized, so confidence intervals need to be built from critical t-values. Like on a normal distribution, a 95% confidence interval is defined by the bounds containing the middle 95% of the data. One student reasonably suggested the following for a 95% confidence interval with 23 degrees of freedom. I really liked the explicit syntax definition of the confidence interval.
Alas, the CAS returned the input. It couldn’t find the answer in that form. Cognizant of the lessons learned above, I suggested reframing the query with an infinite bound.
That gave the proper endpoint, but I was again dissatisfied with the need to alter the input, even though I knew why.
That’s when another of my students spoke up to say that he got the solution to work with the initial commands by including a domain restriction.
Of course! When more than one solution is possible, restrict the bounds to the solution range you want. Then you can use the commands that make sense.
FIXING THE INITIAL APPROACH
That small fix finally gave me the solution to the earlier syntax issue with the golf ball problem. There were two solutions to the initial problem, so if I bounded the output, they could use their intuitive approach and get the answer they needed.
If a mean of 288 yards and a standard deviation of 2.8 yards resulted in 12.7% of the area above 291.2, then it wouldn’t take much of a left shift in the mean to leave just 1% of the area above 291.2. Surely that unknown mean would be no lower than 3 standard deviations below the current 288, somewhere above 280 yards. Adding that single restriction to my students’ original syntax solved their problem.
By encouraging a deep understanding of both the underlying statistical content AND of their CAS tool, students are increasingly able to find creative solutions using flexible methods and expressions intuitive to them. And shouldn’t intellectual strength, creativity, and flexibility be the goals of every learning experience?