&

SAMPLING DISTRIBUTION OF SAMPLE MEANS

“Whenever a large sample of chaotic elements are taken in hand and marshaled in the order of their magnitude, an unsuspected and most beautiful form of regularity proves to have been latent all along.” —Sir Francis Galton

 

·      Statistical inference is the process of drawing conclusions about a population based on a smaller sample drawn at random from the population.

 

·      Statistical inference also applies to the results of randomized comparative experiments, but because the reasoning underlying inference is simpler in the context of sampling from a population, I’ll concentrate on that setting.

 

 

**The central concept in classical statistical inference is the notion of a sampling distribution, which describes how sample results behave if we draw repeated samples of a particular size n from a larger population**

 

·      Of course, in any real application of statistical inference, we draw only one sample of size n, but the possibility of sampling repeatedly— if only in principle— provides the conceptual foundation for deciding how informative our sample is about the population.

 

·      Statistical inference thus might seem like sleight-of-hand: we start with only a sample, but end with a conclusion about the entire population.

 

Recall that:

·       A parameter is a number that describes some aspect of the population.

o     For example, the average family income m in the population of New Jersey is a parameter.

§       Suppose, for arguments sake, that this number

§       is m = $43,236. In a real application, parameters are generally not known.

§       N is for ALL the data

 

·       A statistic is a number that can be calculated from the sample data, without any knowledge of population parameters.

·       n denotes the sample data

 

EXAMPLE:

·      Suppose that we draw a random sample of n = 1000 New Jersey families, and that the average income in these families is $42,586.

o    The sample mean,  = $42, 586, is a statistic.

·      We are usually interested in statistics not for themselves, but because they can tell us something about the population.

·      For example, we might want to use the sample mean family income to estimate the (unknown) population mean income

·      Alternatively, we might want to use the sample mean to determine whether average family income in the population of New Jersey has changed since the Census.

·      These two sorts of applications lead to the two principal classical modes of statistical inference:

 

1) Estimation—giving a numerical estimate, and a likely range, for a parameter. For our sample of n = 1000 New Jersey families, we might be able to say that the true mean income is likely to be $42,586 + or - $2050.

 

2) Hypothesis testing—testing whether a given value is consistent or inconsistent with the data.

 

Random Sampling Techniques:

Why sample?

·    Generating information about a population can be done in one of two ways:

1. Taking a census is collecting the desired information from all the elements comprising the population.

2. Sampling is the process of selecting a subset of elements from a population for the purpose of estimating various population parameters.

·    Sampling is less costly than taking a census in the case of large populations.

·     Lower monetary cost.

·     Lower time cost.

 

There are two types of samples.

1. Random sample:

·      A sample in which every element in the population has the same probability of being selected for inclusion in the sample.

·      This probability is: where P = the probability that an element will be selected; n = sample size; N = number of elements in the population.

·      This type of sample is useful in that it yields unbiased estimates of parameters.

2. Nonrandom sample:

·      A sample in which every element in the population does not have the same probability of being selected for inclusion in the sample.

·      This type of sample can yield biased, and unreliable estimates of parameters.

 

 

And there are two kinds of error.

 

1. Sampling error:

·      The difference between a sample statistic and its corresponding parameter;

·        In a random sample, this error is due to chance; it is measurable and can be analyzed.

 

 

 

2. Nonsampling error:

·      Error due to design and execution mistakes: definitional, missing data, input processing, analysis errors, and the like.

·      This type of error is insidious (dangerous) in that it can be in the data without knowledge of its presence.

·      This type of error is minimized by careful design and execution of the study.

 

 

Four types of random sampling techniques:

·      Simple random sampling- Every person has an equal chance of being selected

·      Systematic sampling- Every kth person is selected

·      Stratified random sampling- divide into groups and then sample

·        Cluster (area) sampling- naturally occurring groups- stats on the groups

 

Simple random sampling: A procedure in which a simple chance process is used to select elements from the population.

·     FISHBOWL TECHNIQUE: Drawing elements from a hat. $ Selection using a table of random numbers.

·     RANDOM NUMBER GENERATORS: Selection using a random number generator in Excel.

 

Example:

Ponzi Chips, a supplier of gaming chips to Nevada casinos, wants to produce quick estimates of the mean and standard deviation of their currently outstanding 90 accounts receivable. To accomplish this, Ponzi wants to take a simple random sample of n=10 of the N=90 such accounts.

The following two-step procedure is used to generate the sample:

1.  The 90 accounts are numbered from 1 to 90. 2. The 10 accounts are selected for the sample using a table of random numbers.

2.  Step 2 could be accomplished using the random number generator in Excel.

 

Sampling Distributions

Sampling distribution:

·       A probability distribution for ALL the possible values of a statistic with a given sample size.

·       ALL the possible samples you can take from a population of  a particular sample size (such as n = 2 or n = 3, etc.)

 

We will consider two sampling distributions:

·       Sampling distribution of the mean,  

·       Sampling distribution of the proportion, p.

 

&

%

Sampling variation

A key fact about sample statistics is that they are random variables that vary from sample to sample.

·      Sampling variability:

o    If we were to select another random sample of 1000 New Jersey families, it is highly unlikely that we would get a sample mean income of exactly $42,586. The variation of a sample statistic from one sample to the next is called sampling variation or sampling variability.

o     When sampling variation is large, the sample contains little info about the population parameters.

o    But when sampling variation is small, the sample statistic is informative about the parameter, even though it is very unlikely that the statistic will be exactly equal to the parameter in a given sample.

 

EXAMPLE OF A SAMPLING DISTRIBUTION:

·       Taking the class as the population, suppose we wish to determine the mean height m (in inches).

 

·       Because the class is relatively small— as populations go— we can obtain the complete distribution of heights of students in the class, and calculate the average height

 

·      That’s the true value of the parameter, m, and the distribution height is the population distribution.

 

But suppose you were limited to sampling only n = 4 students.

 

§           How could you estimate µ?

 

HOW TO SAMPLE:

Here’s where imagination comes in:

·      Imagine we were to draw 50 repeated samples of individuals in the class, each of size n = 4.

 

·      We find the height of each individual in the sample, and

calculate the average height of the people in the sample.

 

·      We calculate the average height in each sample, and look at the distribution of the 50 averages— Ah! we’ve got a sampling distribution of the mean.

·      In this case, the mean height, m, in the class is the parameter (here, m = 65.83) while the mean height  in each sample is the statistic.

 

 

 

 

 

The first 25 samples of size 4:

·       Using a table of random numbers, we can actually draw 50 samples of size n = 4.

 

Here are the first 25 samples I drew, each with the mean height for that sample.

Sample Obs1    Obs2   Obs3   Obs4   Mean x

--------------------------------------------

1      61     68     64     62     63.75

2      62     64     64     62     63.00

3      62     51     67     69     62.25

4      70     68     71     64     68.25

5      69     67     68     68     68.00

6      73     51     51     51     56.50

7      61     61     65     62     62.25

8      69     67     67     68     67.75

9      65     62     62     71     65.00

10     54     66     70     65     63.75

11     67     63     65     64     64.75

12     68     64     63     51     61.50

13     68     68     62     66     66.00

14     71     68     65     62     66.50

15     51     69     63     124    76.75

16     63     65     64     62     63.50

17     62     73     70     64     67.25

18     62     66     63     63     63.50

19     68     68     62     62     65.00

20     63     63     61     73    65.00

21     63     66     69     64     65.50

22     65     69     61     62     64.25

23     62     71     69     68     67.50

24     51     63     65     62     60.25

25     65     65     68     62     65.00

 

 

The three amigos: population, sample, and sampling distributions

 

Notice that there are three distinct distributions here:

 

  1. The population distribution: The distribution of heights in the population.

CHARACTERIZED BY:

·       The mean of this distribution is m = 65.83;

·       The population standard deviation is s = 8.13.

 

N Mean   Std Dev   Minimum   Maximum

---------------------------------------

67 65.83 8.13       51.0       124.0

---------------------------------------

 

  1. The sample distribution:

·       The distribution of heights in a particular sample of size n = 4.

·       The observations are x1 ; x2 ; x3 ; and x4, and the mean of the sample distribution is   .

CHARACTERIZED BY:

·      

·     s

Sample Obs1   Obs2   Obs3   Obs4   Mean

---------------------------------------

1      61     68     64     62     63.75

 

 

 

3. The sampling distribution of the sample means: The distribution of sample means is the collection of sample means for all the possible random samples of a particular size (n) that can be obtained from a population;

A sampling distribution is a distribution of statistics obtained by selecting all the possible samples of a specific size from a population.

$

**The important distribution for statistical inference is the sampling distribution of the sample means. **

 

·       The sampling distribution of a statistic is the distribution of the statistic in all possible samples of the same size drawn from a population.

 

·       When we select 50 samples, the resulting distribution of the statistic only approximates the true sampling distribution.

 

·       Keeping these three distributions separate is one of the keys

    to understanding statistical inference.

·       In other words, what we want to do is look at all of the possible samples (of a particular size, this part is important) and make predictions based on the properties of all of them.

 

·       How do we do this? The same kind of thing that we've done in the past, we essentially find the average of those properties.

 

CHARACTERIZED BY:

·       µ

·       s= Standard Error (of the sampling distribution of sample means)

 ,

·      **The standard deviation of a sampling distribution is called the standard error ( or the ESTIMATED STANDARD ERROR OF THE SAMPLING DISTRIBUTION OF SAMPLE MEANS) —is a measure of the sampling variability of a statistic.**

 

Here, I reveal the most important concept of statistical inference:

If we can determine the characteristics of the sampling distribution—

·      mean

·      variability

·      shape

from statistical theory, then:

 

 

1. We don’t have to generate all those samples to know how a statistic varies.

 

2. We can use the theory to estimate the parameter, or test an hypothesis.

·      Happily, it turns out that we can, for most statistics.

 

3.   is our best estimate of µ because the sample mean is a least squares unbiased predictor of the population mean.

4. Sigma squared, s2, is the variance of the population and s2 is our best estimate of sigma squared s2 .

 

**SO WE NOW NEED TO DISTINGUISH BETWEEN THE POPULATION VARIANCE (AND STANDARD DEVIATION) AND THE SAMPLE VARIANCE (AND STANDARD DEVIATION)**

 

 

 

 

 

 

 

 

 

So, the variance and standard deviation formulas for the Population are as follows:

 

Population Standard Deviation:              Population Variance:

                     

                                                       

                  

 

The variance and standard deviation formulas for a Sample are as follows:

Sample Standard Deviation:               Sample Variance:

                                                                            

                                            

 

                             

For the sample formulas, we replaced

1)                   µ with   

2)                   n with n-1 in the denominator.

 

 

 

 

Why is n replaced with n-1?

 

·      The goal of inferential statistics is to use limited information from samples to draw general conclusions about populations.

 

·      Thus, samples must represent populations from which they came.

 

 

·      Sampling distributions of sample means tend to be less variable than populations. AND

 

·      Whatever the distribution of the data, the sampling distribution of sample means becomes closer to the normal distribution as the sample size increases.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The Sampling distribution of sample means, mx

 

Example: Tossing dice

 

·      Let us consider a simple example: We toss n fair dice; observe the

number of dots showing for each die; and calculate

the mean number of dots x = P

 

·      If, for instance, n = 2 and the sample is x1 = 2; x2 = 3, then

 = (2 + 3)=2 = 5=2 = 2.5.

 

Recall that there are three distributions that we must keep separate in our minds:

1. Population: The distribution of X in the population. In this case,

X has the following probability distribution:

x    probability (p)

1    1/6

2    1/6

3    1/6

4    1/6

5    1/6

6    1/6

1.0

·      The mean of this distribution is = =  3.5

·      and the standard deviation is σ = 1.708

 

2. Sample: The distribution of x in a particular sample, e.g.,

x1 = 2; x2 = 3. The sample mean= 2.5, is the average value of x in this sample.

For real data we could (and normally would) make a frequency distribution or other display of the sample distribution.

 

3. Sampling distribution: The sampling distribution of the mean across all possible samples, µ

·      When n = 2(FROM THE POPULATION ABOVE OF ROLLING one die), there are only 6 x 6 = 36 possible samples, so we can enumerate them all, along with their sample means:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

sample            x1        x2              xbar

1

1

1

1.0

2

1

2

1.5

3

2

1

1.5

4

1

3

2.0

5

2

2

2.0

6

3

1

2.0

7

1

4

2.5

8

2

3

2.5

9

3

2

2.5

10

4

1

2.5

11

1

5

3.0

12

2

4

3.0

13

3

3

3.0

14

4

2

3.0

15

5

1

3.0

16

1

6

3.5

17

2

5

3.5

18

3

4

3.5

19

4

3

3.5

20

5

2

3.5

21

6

1

3.5

22

2

6

4.0

23

3

5

4.0

24

4

4

4.0

25

5

3

4.0

26

6

2

4.0

27

3

6

4.5

28

4

5

4.5

29

5

4

4.5

30

6

3

4.5

31

4

6

5.0

32

5

5

5.0

33

6

4

5.0

34

5

6

5.5

35

6

5

5.5

36

6

6

6.0

 

Each of these samples occurs with equal probability, 1/36, producing the following sampling distribution for sample means, xbar :

 

p

1.0

1/36

1.5

2/36

2.0

3/36

2.5

4/36

3.0

5/36

3.5

6/36

4.0

5/36

4.5

4/36

5.0

3/36

5.5

2/36

6.0

1/36

 

36/36 = 1

·      This distribution has a mean of μ = 3.5 (which is precisely equal to the population mean m

·      The standard deviation of the sample, s =  1.208. Notice that the standard deviation of the sample means is smaller than the standard deviation of the population, s = 1.708. 

·      THIS IS BECAUSE SAMPLE MEANS ARE BASED ON MEANS OF SAMPLES IN THE SAMPLING DISTRIBUTION WHEREAS POPULATIONS ARE BASED ON THE ACTUAL SCORES IN FRONT OF YOU.

 

 

 

 

Suppose that the sample size is increased to n = 3:

Then there are 6 x 6 x 6 = 216 different samples, each of which is chosen with probability 1/216:

sample

X1

X2

X3

Xbar

1

1

1

1

1.0000

2

1

1

2

1.3333

3

1

2

1

1.3333

4

2

1

1

1.3333

5

1

1

3

1.6667

6

1

2

2

1.6667

7

1

3

1

1.6667

8

2

1

2

1.6667

9

2

2

1

1.6667

10

3

1

1

1.6667

. . . . . . . . . . . . . . .

 

 

 

 

211

6

5

5

5.3333

212

6

6

4

5.3333

213

5

6

6

5.6667

214

6

5

6

5.6667

215

6

6

5

5.6667

216

6

6

6

6.0000

 

·       The sampling distribution of the sample means, shown in the left panel below, has mean 3.5 and standard deviation 0.986. The mean of x is still m, but its standard deviation has gotten smaller.

 

·       We could do the same thing for n = 4, giving 6 x 6 x  6 x  6 =1296 possible samples.

 

 

We can summarize these results as follows:

Sample

Size

Mean

Standard deviation

Shape

n = 1

3.50

1.708 = s

uniform

n = 2

3.50

1.208 = s/Ö2

triangular

n = 3

3.50

0.986 = s/Ö3

bell-ish

n = 4

3.50

0.854 = s/Ö4

more bell-ish

 

We see:

·      The mean of the sampling distribution of means (remains the same. SO, m =  = µ 

·      The standard deviation decreases as n increases= a BIASED ESTIMATE BY BEING TOO SMALL, AS WHAT IS GOING ON IN THE POPULATION.

·      The shape becomes closer to the normal distribution.

 

 

 

 

 

 

 

 

 

 

 

 

 

Back to our original question of why n-1 in the denominator of the sample variance and standard deviation?

 

·       Sampling variability is Less variable than Population variability and therefore tends to give a BIASED estimate of population variability.

 

·       THUS, sample means (or the sampling distribution of sample means) tends to UNDERESTIMATE the variability that occurs within the population.

·       To correct for this bias, it is necessary to adjust the calculation of the variance and standard deviation when you are dealing with samples.

 

·       Placing n-1 in the denominator instead of n corrects the sample back towards the population.

 

 

·       Because dividing by n-1 instead of by n produces a larger result and makes the sampling variability an accurate, UNBIASED ESTIMATE of the population variability

 

·       Thus, the sample variance and standard deviations are referred to as unbiased estimates

 

 

 

·      Remember that each of our sample standard deviations are not very good reflections of the population (they are smaller, and thus are biased estimates of the population mean).

 

·      So what we find (well, the mathematicians, not you and me) is that sampling distributions vary not according to standard deviation, but according to the standard error of the means!

·      That is, we (huh, the mathematicians) found the standard error to be a better measure of variability of sampling distributions than the standard deviation.

 

·      The mathematicians discovered that if you take the sample standard deviation and divide it by the square root of the sample size (n), you get a much better estimate of the true variability between sample means and the true population mean.

 

·      The standard error then tells us how typical or unusual a sample mean is given sampling error! By typical we mean in the central area or hump and by unusual, we mean in the "tails" of the distribution.

·       Just like an I.Q. of 155 is unusual, a group of randomly selected 10 people whose mean I.Q. is 155 would be very unusual.

·       In fact it would be so unusual we would think something wrong had happened and begin to look for the explanation!

·       Notice that even a mean of 130 would be unusual, about 2.28%! (LOOK AT Z DISTRIBUTION)

·       To get a mean of 130 based on a group of people you would have to have about half the group, 5, with I.Q.'s over 130!

 

And we can see that they are only 2.28% of the people out there! For 5 of them to show up in your small sample of 10 people would be unusual indeed, making you think that 50% of the people had IQ's of 130 or better.

 

And if your sample was not from the population, but selected from people who lived under power lines, you might think power lines were responsible for the large number of high IQ people in your sample. You might be right and you might be wrong

 

Looking ahead: Statistical inference

 

All this stuff about dice, and means samples of size n = 4 from the distribution of heights in the class was just designed to make the idea of a sampling distribution concrete.

 

Here is how these ideas are used in statistics:

·       Assume we know (somehow) that the average income of all New Jersey families in 1990 was  = $45,000 and the population standard deviation was = $16,000.

 

·       We want to know if the average income has changed by the year 2000, so we collect a well-designed random sample of n = 900 people and (somehow) measure their current income (adjusting for inflation).

 

 

·       We find that  = $49,580 in our sample. Is this result consistent with the hypothesis that the population mean is the same as in 1990?

 

·      We can answer this question by finding the probability that the  of  $49,580, = m of $45,000

 

·      If this is true, then the sample mean  should have a normal distribution with a mean of $45,000, and a standard deviation, s =  16000/√900 = 533.33

 

 

5 Properties of the Sampling Distribution

1) Consequently, as the sample size grows, tends to get closer and closer to the µ  In a very large sample, x almost certainly will be very close to µ. This is called the law of large numbers.

 

2)  Regardless of the shape of the population distribution, the sampling distribution of sample means is approximately normal,

 

3) With the approximation improving as the sample size grows. This result is called the central limit theorem.

 

4) How large n needs to be for the approximation to be good enough depends upon how far from normal the population distribution is,

·      but n of  100 almost always suffices, and n of 30 is usually sufficient for most real data.

 

5)  If the population distribution of is itself normal, then, regardless of n, the sampling distribution of samples means will also be normal and even an n = 1 will suffice.

 

 

 

SUMMARY:

1) The sampling distribution is a theoretical distribution of a sample statistic

 

2) There is a different sampling distribution for each sample statistic

 

3) Each sampling distribution is characterized by parameters, two of which are µx  and s

 

4) The sampling distribution of the mean is a special case of the sampling distribution

 

5) The Central Limit Theorem relates the parameters of the sampling distribution of the mean to the population mean

 

CENTRAL LIMIT THEOREM:

 

1) The mean of the sampling distribution of the mean, µx, equals the mean of the population, μ

 

2) The “standard error” of the mean (“SEM”) or “Standard error of the sampling distribution of sample means!” or just plain old STANDARD ERROR, σx, equals the standard deviation of the population, σ, divided by the square root of N

 

3) As sample size increases, the sampling distribution of the sample means will approach a normal distribution- regardless of the shape of the parent distribution.