The inclusion of probability and statistics as content standards in the National Council of Teachers of Mathematics standards documents (NCTM, 1989, 2000) triggered increased interest in how students learn statistics. The broad goal for instruction is for students to be able to reason statistically in order to become informed citizens and intelligent consumers. This goal implies that students should be able to analyze sample data and make subsequent decisions about the population from which the sample was drawn. Such reasoning is at the heart of understanding statistical inference.

Related Literature and Theoretical Background

            Statistics education research over the past two decades has investigated the difficulties students encounter in drawing appropriate conclusions about populations from sample data. In particular, students’ difficulties with formal inferential reasoning have been well-documented. Results of the Comprehensive Assessment of Outcomes in Statistics (CAOS 4), administered to 763 introductory statistics students, found students scored low on the pre/posttest questions about interpreting P-values (delMas, Garfield, Ooms, and Chance, 2007). Another study revealed undergraduate students at the completion of introductory statistics courses lacked an understanding of exactly what the result of a hypothesis test means and confused the probabilistic meanings of P-value and significance level (Castro Sotos, Vanhoof, Noortgate & Onghena, 2009). Many researchers trace these difficulties back to the basic concepts of variation and distribution. Variation, a key concept in statistical thinking, involves coming to terms with the implicit uncertainty contained in data (Wild and Pfannkuch, 1999).

            Researchers have begun to study informal inferential reasoning which brings together the concepts of variation, distribution, and sampling before the procedures of formal inference are introduced. Informal inferential reasoning involves students drawing conclusions about populations based on data, but without the statistical computations of formal inference. Makar and Rubin (2009) described essential features of informal inferential reasoning as using data as evidence to make generalizations while acknowledging a level of certainty. Students’ informal inferential reasoning is thought to be a precursor to the development of formal inferential reasoning which requires interpreting confidence intervals and hypothesis tests. Thus, current research efforts are focused on defining the key elements of informal inferential reasoning.

            Toward this effort, Zieffler, delMas, Garfield, and Reading (2008) developed a framework consisting of three types of tasks that can be used to study students' informal inferential reasoning:

            Type 1: Estimate and draw a graph of a population based on a sample;

            Type 2: Compare two or more samples of data to infer whether there is a real difference between the populations from which they were sampled; and

            Type 3: Judge which of two competing models or statements is more likely to be true. (p.47)

Using this framework, this research study examined the relationship between college students’ informal and formal inferential reasoning.  The research questions for this study were:

            1) How do students' informal and formal inferential reasoning change over a semester of formal inferential statistics instruction?

            2) What is the relationship between students' informal inferential reasoning and their formal inferential reasoning?

Methodology and Data Analysis

            This was a pilot study with post-secondary students taking a second course in introductory statistics. The students had taken a first course in statistics which covered sampling theory, variation, and distribution. The second course draws on this knowledge to develop formal inferential reasoning. Hence, the students in this study should have some prior understanding of describing distributions of data, recognizing variation in populations and samples, and chance variation and be poised to develop more formal inferential reasoning over the course of a semester of instruction. Based on the Zieffler et al. (2008) framework, we designed a pre/posttest that included both formal and informal inferential reasoning questions. We also designed a task-based interview protocol (Goldin, 2000) to provide the opportunity for students to articulate their evidenced-based arguments for each type of informal reasoning task and for formal inferential reasoning tasks. It was hypothesized that students exhibiting strong informal inferential reasoning would show improvement on formal inferential reasoning tasks with a semester of instruction. 

            The pre/posttest was given to 14 students in a lecture style second course in introductory statistics. Four student volunteers were chosen to participate in the task-based interviews. Matched pairs t-tests were conducted to determine if significant changes occurred in students' informal, formal, and overall inferential reasoning. Correlations were examined to determine if the type of informal inferential reasoning task (Zieffler et al., 2008) was related to formal inferential reasoning. Several regression models were examined as well to determine the influence of each type of informal inferential reasoning task on formal inferential reasoning. The task-based interviews were analyzed in accordance with the Zieffler et al. (2008) framework.

Results and Conclusions

            Matched pairs t-tests of pre/posttest scores revealed no improvement in the overall, informal, or formal inferential reasoning scores. This result confirms the findings of other research that traditional instruction does not support students’ inferential reasoning. However, posttest results did show a significant correlation between Type 3 informal inferential reasoning (judging between competing models) and formal inferential reasoning. Together Type 2 (comparing two samples of data) and Type 3 informal inferential reasoning accounted for approximately 61% of the variance in formal inferential reasoning on the posttest. In addition, students' responses during the task-based interviews indicated a positive relationship between the Type 1(estimating and drawing a population graph) and Type 3 tasks and formal inferential reasoning.

            Hence, student performance on the informal Type 1 tasks, estimating and drawing a population graph, and the informal Type 3 tasks, judging between competing models, were positively related to student performance on formal inferential reasoning questions. These results provide tentative support for the hypothesis by indicating a positive relationship between types of informal inferential reasoning and formal inferential reasoning. Our conclusions are tentative given the limitations of the small sample size. Our results may also be an indication that the tasks based on the Zieffler et al. (2008) framework may not fully capture important dimensions of informal inferential reasoning. Investigating the dimensions of that reasoning is part of our ongoing research.