Sample size determination or estimation is the act of choosing the number of observations or replicates to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample. In practice, the sample size used in a study is usually determined based on the cost, time, or convenience of collecting the data, and the need for it to offer sufficient statistical power. In complex studies, different sample sizes may be allocated, such as in stratified surveys or experimental designs with multiple treatment groups. In a census, data is sought for an entire population, hence the intended sample size is equal to the population. In experimental design, where a study may be divided into different treatment groups, there may be different sample sizes for each group.
Sample sizes may be chosen in several ways: using experience – small samples, though sometimes unavoidable, can result in wide confidence intervals and risk of errors in statistical hypothesis testing. using a target variance for an estimate to be derived from the sample eventually obtained, i.e., if a high precision is required (narrow confidence interval) this translates to a low target variance of the estimator. the use of a power target, i.e. the power of statistical test to be applied once the sample is collected. using a confidence level, i.e. the larger the required confidence level, the larger the sample size (given a constant precision requirement).
Introduction
Sample size determination is a crucial aspect of research methodology that plays a significant role in ensuring the reliability and validity of study findings. In order to influence the accuracy of estimates, the power of statistical tests, and the general robustness of the research findings, it entails carefully choosing the number of participants or data points to be included in a study.
Consider the case where we are conducting a survey to determine the average satisfaction level of customers regarding a new product. To determine an appropriate sample size, we need to consider factors such as the desired level of confidence, margin of error, and variability in the responses. We might decide that we want a 95% confidence level, meaning we are 95% confident that the true average satisfaction level falls within the calculated range. We also decide on a margin of error, of ±3%, which indicates the acceptable range of difference between our sample estimate and the true population parameter. Additionally, we may have some idea of the expected variability in satisfaction levels based on previous data or assumptions.
Importance Larger sample sizes generally lead to increased precision when estimating unknown parameters. For instance, to accurately determine the prevalence of pathogen infection in a specific species of fish, it is preferable to examine a sample of 200 fish rather than 100 fish. Several fundamental facts of mathematical statistics describe this phenomenon, including the law of large numbers and the central limit theorem.
In some situations, the increase in precision for larger sample sizes is minimal, or even non-existent. This can result from the presence of systematic errors or strong dependence in the data, or if the data follows a heavy-tailed distribution, or because the data is strongly dependent or biased.
Sample sizes may be evaluated by the quality of the resulting estimates, as follows. It is usually determined on the basis of the cost, time or convenience of data collection and the need for sufficient statistical power. For example, if a proportion is being estimated, one may wish to have the 95% confidence interval be less than 0.06 units wide. Alternatively, sample size may be assessed based on the power of a hypothesis test. For example, if we are comparing the support for a certain political candidate among women with the support for that candidate among men, we may wish to have 80% power to detect a difference in the support levels of 0.04 units.
Estimation Estimation of a proportion
A relatively simple situation is estimation of a proportion. It is a fundamental aspect of statistical analysis, particularly when gauging the prevalence of a specific characteristic within a population. For example, we may wish to estimate the proportion of residents in a community who are at least 65 years old.
The estimator of a proportion is where X is the number of 'positive' instances (e.g., the number of people out of the n sampled people who are at least 65 years old). When the observations are independent, this estimator has a (scaled) binomial distribution (and is also the sample mean of data from a Bernoulli distribution). The maximum variance of this distribution is 0.25, which occurs when the true parameter is p = 0.5. In practical applications, where the true parameter p is unknown, the maximum variance is often employed for sample size assessments. If a reasonable estimate for p is known the quantity may be used in place of 0.25.
As the sample size n grows sufficiently large, the distribution of will be closely approximated by a normal distribution.NIST/SEMATECH, "7.2.4.2. Sample sizes required", e-Handbook of Statistical Methods. Using this and the Wald method for the binomial distribution, yields a confidence interval, with Z representing the standard Z-score for the desired confidence level (e.g., 1.96 for a 95% confidence interval), in the form:
To determine an appropriate sample size n for estimating proportions, the equation below can be solved, where W represents the desired width of the confidence interval. The resulting sample size formula, is often applied with a conservative estimate of p (e.g., 0.5):
for n, yielding the sample size
in the case of using 0.5 as the most conservative estimate of the proportion. (Note: W/2 = margin of error.)
Otherwise, the formula would be which yields
In the right-hand figure one can observe how sample sizes for binomial proportions change given different confidence levels and margins of error.
For example, in estimating the proportion of the U.S. population supporting a presidential candidate with a 95% confidence interval width of 2 percentage points (0.02), a sample size of (1.96)2/ (0.022) = 9604 is required with the margin of error in this case is 1 percentage point. It is reasonable to use the 0.5 estimate for p in this case because the presidential races are often close to 50/50, and it is also prudent to use a conservative estimate. The margin of error in this case is 1 percentage point (half of 0.02).
In practice, the formula : is commonly used to form a 95% confidence interval for the true proportion. The equation can be solved for n, providing a minimum sample size needed to meet the desired margin of error W. The foregoing is commonly simplified:"Confidence Interval for a Proportion" n = 4/W2 = 1/B2 where B is the error bound on the estimate, i.e., the estimate is usually given as within ± B. For B = 10% one requires n = 100, for B = 5% one needs n = 400, for B = 3% the requirement approximates to n = 1000, while for B = 1% a sample size of n = 10000 is required. These numbers are quoted often in news reports of opinion polls and other sample surveys. However, the results reported may not be the exact value as numbers are preferably rounded up. Knowing that the value of the n is the minimum number of sample points needed to acquire the desired result, the number of respondents then must lie on or above the minimum.
Estimation of a mean Simply speaking, if we are trying to estimate the average time it takes for people to commute to work in a city. Instead of surveying the entire population, you can take a random sample of 100 individuals, record their commute times, and then calculate the mean (average) commute time for that sample. For example, person 1 takes 25 minutes, person 2 takes 30 minutes, ..., person 100 takes 20 minutes. Add up all the commute times and divide by the number of people in the sample (100 in this case). The result would be your estimate of the mean commute time for the entire population. This method is practical when it's not feasible to measure everyone in the population, and it provides a reasonable approximation based on a representative sample.
In a precisely mathematical way, when estimating the population mean using an independent and identically distributed (iid) sample of size n, where each data value has variance σ2, the standard error of the sample mean is:
This expression describes quantitatively how the estimate becomes more precise as the sample size increases. Using the central limit theorem to justify approximating the sample mean with a normal distribution yields a confidence interval of the form
, where Z is a standard Z-score for the desired level of confidence (1.96 for a 95% confidence interval).
To determine the sample size n required for a confidence interval of width W, with W/2 as the margin of error on each side of the sample mean, the equation
can be solved. This yields the sample size formula, for n:
.
For instance, if estimating the effect of a drug on blood pressure with a 95% confidence interval that is six units wide, and the known standard deviation of blood pressure in the population is 15, the required sample size would be , which would be rounded up to 97, since sample sizes must be integers and must meet or exceed the calculated minimum value. Understanding these calculations is essential for researchers designing studies to accurately estimate population means within a desired level of confidence.
Required sample sizes for hypothesis tests One of the prevalent challenges faced by statisticians revolves around the task of calculating the sample size needed to attain a specified statistical power for a test, all while maintaining a pre-determined Type I error rate α, which signifies the level of significance in hypothesis testing. It yields a certain power for a test, given a predetermined. As follows, this can be estimated by pre-determined tables for certain values, by formulas, by simulation, by Mead's resource equation, or by the cumulative distribution function:
Tables Power Cohen's d 0.2 0.5 0.8 0.25 84 14 6 0.50 193 32 13 0.60 246 40 16 0.70 310 50 20 0.80 393 64 26 0.90 526 85 34 0.95 651 105 42 0.99 920 148 58 The table shown on the right can be used in a two-sample t-test to estimate the sample sizes of an experimental group and a control group that are of equal size, that is, the total number of individuals in the trial is twice that of the number given, and the desired significance level is 0.05.Chapter 13, page 215, in: The parameters used are: The desired statistical power of the trial, shown in column to the left. Cohen's d (= effect size), which is the expected difference between the means of the target values between the experimental group and the control group, divided by the expected standard deviation.
Formulas Calculating a required sample size is often not easy since the distribution of the test statistic under the alternative hypothesis of interest is usually hard to work with. Approximate sample size formulas for specific problems are available - some general references are Cohen, J. (1987), Statistical Power Analysis, 2nd edition, Hillsdale(NJ):Lawrence Erlbaum Associates, Inc. and Desu, M.M. and Raghavarao, D. (1990), Sample Size Methodology, New York:Academic Press.
A computational approach (QuickSize) The QuickSize algorithm Amaratunga, D. (1999). Searching for the right sample size. The American Statistician, 53(1), 52–55. https://doi.org/10.2307/2685652 is a very general approach that is simple to use yet versatile enough to give an exact solution for a broad range of problems. It uses simulation together with a search algorithm.
Mead's resource equation