3 Data distributions
3.1 Describing observed data
When we observe data, we can describe it by its central tendency and dispersion. However, it is also often useful to describe the distribution of the data graphically to get a sense of its shape. A histogram can help us with this representation. A histogram divides a numeric variable into bins and counts the number of observations in each bin. The number of observations in each bin is then displayed as a bar plot, with each bar representing a bin (Figure 3.1)1. In Figure 3.1, a separate histogram has been created for each gender in the data set.
1 In JASP, the Descriptive Statistics module lets us create a basic histogram; however, in JASP it is called Distribution plots with the possibility to add density estimates.
An alternative to the histogram is a density plot. The density plot shows us the relative density of observations along the scale of the variable of interest. Often, the curve shows a smoothed estimate of the distribution (Figure 3.2). In a density plot, the y-axis is sometimes omitted as they are not interpretable as counts. Instead the are under a density curve sums 1.
An alternative approach to summarise a distribution could be a boxplot (Figure 3.3). A boxplot summarises the data by displaying the median, the first and third quartile (the box). Whiskers covers observations within 1.5 × the inter-quartile range from the box. Observations outsides the whiskers are displayed using points.
Categories can also be summarised as using frequencies. In Figure 3.4 we have categorized age in bins creating an ordered categorical variable (ordinal), a second categorical variable, gender (nominal), let’s us compare the number of observations between Males and Females within each age group.
The shape of the data is of interest to us (Sokal and Rohlf 2012, 19). For example, in the above figures we can notice that only a very small proportion of the competitors approach the 2 hour mark.2 The distribution is skewed to the right and we might discover some unusual peaks in the histogram at three and four hours. Additionally, we see a slight over-representation of women in the age category 18-29. Patterns like these in the data could lead us to further investigate, for example, the limits of human performance (why no sub 2 hour times?), the psychology of specific run times (why do more runners finish at sub 3 hours?) or the general demographics of the participants (the tail of the distribution and gender differences in participation).
Sokal, Robert R., and F. James Rohlf. 2012. Biometry : The Principles and Practice of Statistics in Biological Research. [Extensively rev.] 4th. W.H. Freeman.
2 The marathon world records are 2:00:35 and 2:09:56, for men and women, respectively (source).
3.2 Theoretical data distributions
Above, we have considered the distribution of data that we might observe, such as marathon finishing times. We can call this the empirical data distribution of some quantity of interest (Altman and Bland 1995). In many cases, such observed, empirical distributions follow distinct patterns that seem to repeat in many natural situations. Why?
Altman, Douglas G., and J. Martin Bland. 1995. “The Normal Distribution.” BMJ: British Medical Journal 310 (6975): 298–98. https://www.jstor.org/stable/29726236.
Shinohara, Kikunori, Akitaka Yanagisawa, Yutaka Kagota, et al. 1999. “Physiological Changes in Pachinko Players; Beta-endorphin, Catecholamines, Immune System Substances and Heart Rate.” APPLIED HUMAN SCIENCE Journal of Physiological Anthropology 18 (2): 37–42. https://doi.org/10.2114/jpa.18.37.
Consider Pachinko, the Japanese mechanical gambling machine that is both addictive and stress-inducing (Shinohara et al. 1999). In a basic form, the machine allows its user to set a ball in motion towards a tilted or vertical grid of pegs (Figure 3.5). The ball bounces through the grid so that at each peg it either goes to the right or to the left. At the bottom of each machine, slots with a specific payout are organized so that the smallest payout comes from the slots where balls are most likely to end up. In contrast, the highest payout comes from slots located where balls are least likely to end up. A gambler could, by chance, for example, win the top prize if a ball takes a very unlikely route through the grid.
Any individual ball’s journey through the grid is impossible to predict (bad for the gambler). Still, the overall pattern is very predictable (good news for the owner of the machine). In Figure 3.6, we simulate 1000 balls dropped through a Pachinko machine3 where each orange line represents a potential route through the grid. As there are many more ways for a ball to end up in the center slots, the bell-shaped distribution of balls emerges as a result.
3 This version of a Pachinko machine looks like a Galton board used to illustrate how the familiar bell-shape can emerge from summing up many random events.
It turns out that the resulting bell-shaped distribution is easily predicted by a theoretical distribution called the normal distribution. The fact that the resulting distribution is predictable is relevant for the owners of the Pachinko parlors. When designing the Pachinko machine, the payout can be balanced with respect to the overall theoretical distribution in a way that guarantees that the house always wins, in the long run.
Above, we made a simplification: at each peg, there is a limited number of possibilities for the ball’s route. It can either go right or left. If we redo the simulation and instead imagine that there is an unlimited number of possibilities between -1 and 1 at each “row” in the Pachinko board, the resulting distribution of the balls’ location will be even smoother (Figure 3.7).
Why is the pattern predictable? This is not easily explained (Lyon 2014). But the Pachinko machine can help us develop the intuition. Each peg in the machine gives the ball a 50% chance of going to the right or to the left, respectively. When the ball moves through rows of pegs, we effectively add up small perturbations to the ball’s trajectory. An extreme route through the pegs could be that the ball ends up going to the left at all pegs. In a grid with 10 rows, this would result in a probability equivalent to \(0.5^{10} \approx 0.00098\), highly unlikely. However, a ball could end up in the center slot in so many more ways. Multiple combinations of left and right turns end up in the center slot. Still, only one combination of turns results in the farthest left slot (Figure 3.6). This means that out of a set of balls, most will end up in slots that have more possible combinations of left and right turns. Similarly, when drawing a random number between -1 and 1 (Figure 3.7), extreme totals requires that all draws come from near the boundary of the interval.
4 In our example simulation (Figure 3.7), we have at each step randomly drawn a number from a uniform distribution going from -1 to 1. It turns out that the variance of a uniform distribution is \(=\frac{(b-a)^2}{12}\) which in our case is \(=\frac{(1-(-1))^2}{12} = 0.3333\). We can add variances to get the variance of the distribution of sums, so for 10 steps we get \(0.333 \times 10 = 3.333\). Going from variances to standard deviations follows \(\operatorname{SD} = \sqrt{\operatorname{Var}}\). This gives us the theoretical standard deviation of the resulting distribution \(\sqrt{3.333} = 1.826\).
The bell-shaped curve, which can be described using the theoretical “normal” distribution, emerges because if a distribution has a fixed mean and standard deviation, the bell-shape represents the distribution with the least amount of extra information other than the mean and standard deviation (Lyon 2014). Given the way we have simulated ball positions in Figure 3.7, we know that the mean of the resulting distribution will be 0 and that the variances4 will add up from each step in the simulation. We can therefore create the theoretical probability distribution using the normal distribution, which contains these characteristics: the least structured distribution given a fixed mean and standard deviation (Figure 3.7).
So, in conclusion, when we observe natural phenomena that are the result of many random processes, we often observe a bell-shaped data distribution. The theoretical distribution that most closely models such a data-generating process is the normal distribution because it is the distribution that contains no other information than a given mean and standard deviation (Lyon 2014).
Lyon, Aidan. 2014. “Why Are Normal Distributions Normal?” The British Journal for the Philosophy of Science 65 (3): 621–49. https://www.jstor.org/stable/26398398.
3.3 The normal model of the world
The normal distribution, that is the theoretical distribution that describes the result of a process where many small, unobserved effects are summed up in each observation, is a very useful distribution. In fact, we use it all the time, even we are not trying to describe this process (see (McElreath 2020, 71) for a discussion).
McElreath, Richard. 2020. Statistical Rethinking: A Bayesian Course with Examples in R and Stan. Second edition. Chapman & Hall/CRC Texts in Statistical Science Series. CRC Press.
UNDER CONSTRUCTION