Remember, hypothesis testing works by assuming a value of a population parameter (the null hypothesis) and calculating how likely you are to see your associated sample statistic (or an even more severe value given by your alternative hypothesis) on any given sample. So let’s say we have a sample statistic like a mean or proportion. How do we calculate the likelihood or probability that we would observe a sample statistic this severe or more? It all comes down to the details of the relevant sampling distribution.
What is a Sampling Distribution?
A sampling distribution is a type of probability distribution that arises in the situation where you are trying to determine the probability of a sample statistic having a specific value or range of values. So for instance, let’s say we want to answer the question, “What is the probability that any given sample of n from a population has a sample average of or more?” There is a way to figure this out if you know the details of the distribution in the population that the sample comes from, or if certain conditions are met with the sample.

So let’s focus on a sampling distribution for a specific sample statistic. Let’s start by talking about the sampling distribution for sample means of a certain sample size from a population. First of all, let’s understand that a sample mean from a specific population is itself a random variable. If you have a population of people whose heights follow a specific probability distribution, an individual random person’s height is a random variable since it can be various values with a different associated probability for each value. We can call this variable that represents a random person’s height . But what about a sample mean from a sample of 40 people? Well that is also a random variable. If you take a simple random sample of 40 people from this population and measure their heights, and find the average height in that sample, is it perfectly predictable? No. But, as we shall see an average of 40 people’s heights from this population is much more predictable than any individual person’s height, meaning it has a much lower variance or standard deviation.
But what does the distribution of sample means from a sample of size look like, and what characteristics does it have (like mean and standard deviation)? It turns out that if certain conditions are met, the probability distribution of sample means of samples of size is approximately normal with a mean of and a standard deviation of where is the standard deviation of the variable in the population distribution of . In other words, if we were to take a sample of size from a population that has some randomly distributed variable, we can calculate the probability of the sample having a sample mean above some value or between two values by any method you normally use to find the probability that a Normally distributed variable obtains a specific range of values (think of z-scores and normcdf on your calculator and all that jazz).
Let’s look at the parameters of this sampling distribution of means in a little more detail. The mean of the sample means is the mean of the random variable in the population. So if we had a population of people and the height of a random person can be treated as a continuous random variable, the population has a certain mean height of . That is, the average of all the heights in the population is . What is the average value of all sample means of a specific sample size ? It’s also . In other words, the mean of the sample means is the mean of the random variable in the population. Another way to phrase this is, “the average of all the averages is the average.” This shouldn’t be a particularly shocking result. What is much more interesting is what the standard deviation ends up being.
First, let’s discuss what we’d expect. What do you think is more predictable: the height of a single random person from the population, or the average of 50 random persons’ heights from the population? It shouldn’t take too much explaining to see that the average of 50 people would be more predictable. Why? Because rare events start to average out with a bigger sample size. Is it fairly likely to obtain a single person with a weird height? It’s probably not that unlikely. But if you get one weird height (by weird, I mean a height in a low probability interval of heights) out of 50, will that make the average that unpredictable? No, that one weird height will get averaged into the other 49, and as long as you don’t have many weird heights in that one sample, the average will not be that different than the true average in the population. Even two or three weird heights probably wouldn’t change the average of 50 that much from being around the true average. In other words, in samples, the sample average is more predictable than any single random element from the population. This is reflected in the standard deviation of sample means (). If the standard deviation of the random variable in the population is , then the standard deviation of the sampling distribution of sample means has a standard deviation of . So if you look at the math, the larger the sample size, the smaller the standard deviation of the sampling distribution of means. And if a random variable has a small standard deviation, then that means the values are, on average, not very far from the mean. Therefore, the values of the variable are more predictable. They are all huddled around the mean.

So now that we’ve given an example of what a sampling distribution is, we’ll look at the details of some common sampling distributions.
Sampling Distribution of a Single Population’s Sample Means
This is what we’ve already been looking at in the previous example. It turns out that if certain conditions are met, the sample mean of a sample of size behaves as a random variable, meaning it can take on different values at varying probabilities. The original population distribution and the parameters (like mean and standard deviation) allow us to predict the probabilities of what the various sample means can be. But this predictive power is only true if certain conditions are met. So what are those conditions?
Condition 1: The sample has to be a simple random sample
This condition will need to be met for most probability and inferential statistics questions. Basically, in order for the probability distributions we have to mean anything, we can’t introduce any type of bias or change the probabilities in any way. If we do, we can’t use the same probability distributions.
Condition 2: The sample size cannot be small
Why can the sample size not be too small? If the sample size is too small, we can’t assume that the sampling distribution of means is normally distributed (or basically predictable in any way). We would only be able to proceed if we knew that the distribution of the original variable in the population follows a normal distribution. If the sample size is not too small, the variable that represents the sample means can not be assumed to be normally distributed. For means, that magic number is 30. If the sample size is at least 30, the central limit theorem shows us that the sample average will follow a normal distribution.
Condition 3: The sample size cannot be too big
One of the assumptions built into the math that we are going to use to analyze sampling distributions is that the elements in the sample are independent of one another. This would definitely be true if we were sampling with replacement. However, when sampling, you typically don’t sample with replacement. But if the population is large compared to the sample size, the probabilities are not very different from sampling with replacement. Therefore, the sample elements can be treated as if they were sampled with replacement. The magic number for this condition is 10%. If the sample size is less than 10% of the population size, the probabilities don’t change that much as you sample, and the elements in your sample can be treated as if they were sampled with replacement (independent).
If these conditions are met, the sampling distribution of means can be modeled as a normal distribution with the following parameters:

As you’ll see later, one of the main reasons you learn about sampling distributions is to test predictions about population parameters. In those situations, you rarely know the population standard deviation. As you can see in the above formula, you need to know the population standard deviation to determine the sampling distribution standard deviation and then calculate probabilities. When the population standard deviation is not known, you use the next best thing: the sample standard deviation. In the formula above, you just replace the population standard deviation with the sample standard deviation. Then, the distribution of normalized sample means follows a t-distribution with degrees of freedom.
Therefore, when the population standard deviation is not known, the normalized sample mean random variable can be modeled as:
Sampling Distribution of the Difference of Two Sample Means from Two Different Populations
Now it turns out that a linear combination of sample means from two (or more for that matter) populations also follows a normal distribution. Where this will show up later in hypothesis tests is when you are interested in the value of the difference between two means between two different populations. So that’s what we will focus on here: the sampling distribution of the difference between two sample means. So what conditions for inference need to be met? The same conditions for a single sample mean, but you have to check it for both samples. If both samples meet those conditions for means that we discussed in the previous paragraphs, then the difference in sample means can be modeled as a Normal Distribution with the following parameters:

Just like before with the discussion of the sample mean sampling distribution, the standard deviations in the population are rarely known. Therefore, we just replace the standard deviations of the population with the next best thing: the sample standard deviations. Once we do this, the normalized variable follows the t-distribution:
Be aware that this notation means the degrees of freedom is the minimum of the two degrees of freedom from the two samples.
Sampling Distribution of a Single Population’s Sample Proportions
This distribution arises when you are taking samples of a certain size and you are trying to predict the proportion of observational units within the sample that take on the value of a certain variable. That variable would have to be a binary categorical variable (it’s a categorical variable with either a success or failure level). So it turns out that if certain conditions are met, the sample statistic p_hat can be modeled as a random variable with a probability distribution approximated by a normal distribution. What are those conditions? Well two of the conditions are the same as they were for means. That is, the sample size can not be too big or it will violate the assumption of independent sampling, and the sample must be from random sampling. The last condition is the same as well, in that the sample size can’t be too small. However, what counts as “too small” is different for proportions.
Condition 2: The sample size cannot be too small
This number is different because it doesn’t work on the same principle of the Central Limit Theorem which is used for means. For proportions, the sampling distribution of sample proportions can only be modeled by a Normal Distribution because it actually follows a linear transformation of a Binomial Distribution, but a Binomial Distribution can be approximated as a Normal Distribution if the sample size is large enough. This is the golden number that determines if the sample size is large enough for a sampling distribution of sample proportions can be modeled as a Normal Distribution. and have to both be at least 10. So as you can see, you can get away with taking a relatively small sample if the proportions of success and failure of the variable in the population are both relatively big (around 50%). However, if either the probability of success or failure is very small (the other would then be very big), you will need a larger sample size so that those products of sample size and proportions are both at least 10.

Sampling Distribution of the Difference Between Two Sample Proportions from Two Different Populations
Just like the sampling distribution of the difference between two sample means is just an extension of the sampling distribution of a single mean, the sampling distribution of a linear combination of two sample proportions is just an extension of the sampling distribution of a single proportion. So all the conditions that must be true for a single sample proportion for its sampling distribution to be approximated with a normal distribution must also be true for both samples when we are determining the sampling distribution of a linear combination of sample proportions. Just like for a linear combination of means, we are most interested in specifically the difference between two population proportions. So that is what we will focus on here. If all the conditions discussed previously for proportions are met for both samples, the difference between to sample proportions can be modeled as a Normal Distribution with the following parameters:

What’s the Point?
The point of all of this discussion of sampling distributions is so that we can determine probabilities that sample statistics (means, proportions, or differences of these between two samples) take on certain ranges of values. For instance, if you knew the probability of or the proportion of people in a population who are left-handed, and the conditions for inference of a sample proportion as we discussed earlier are met, we can calculate the probability of getting a sample proportion of left-handed people greater than a certain amount by using the details of the normal distribution that it follows. The following video instructions will make this more clear.
However, the real point of all of this (as far as you’re concerned in this class) is that we can use these ideas of sampling distributions to indirectly test certain assumptions about the population distribution parameters. This process of testing certain values of population parameters using sample data and corresponding sample statistics and their corresponding sampling distributions is called Hypothesis Testing. We’ll explain how that all works in the next blog post.
Leave a Reply