Five Number Summary

In the realm of statistics, summarizing data is essential for gaining insights and making informed decisions. One powerful technique for summarizing numerical data is the Five Number Summary. Whether you’re a data scientist, a business analyst, or a student learning statistics, understanding this summary is crucial.

What is the Five Number Summary?

The Five Number Summary is a statistical summary technique that provides a concise description of the distribution of a dataset. It consists of five key values that divide the data into four equal parts, allowing for a quick understanding of the dataset’s central tendency, spread, and skewness. The components of the Five Number Summary are:

1. Minimum: The smallest value in the dataset.
2. First Quartile (Q1): The value below which 25% of the data falls.
3. Median (Second Quartile, Q2): The middle value of the dataset, separating the lower 50% from the upper 50%.
4. Third Quartile (Q3): The value below which 75% of the data falls.
5. Maximum: The largest value in the dataset.

Visual Representation:
To better comprehend the Five Number Summary, let’s visualize it using a boxplot, also known as a box-and-whisker plot. In a boxplot, the median is represented by a vertical line inside a box, which extends from the first quartile (Q1) to the third quartile (Q3). The whiskers extend from the minimum to the maximum value, encompassing the bulk of the data.

Significance and Interpretation:

The Five Number Summary offers several advantages:

1. Concise Description: Instead of analyzing every single data point, the Five Number Summary provides a succinct overview of the dataset’s distribution, allowing for quick interpretation.
2. Robustness: Unlike measures such as the mean and standard deviation, which can be heavily influenced by outliers, the Five Number Summary is resistant to extreme values, making it robust for skewed or non-normally distributed data.
3. Comparison: It facilitates easy comparison between different datasets, enabling analysts to assess similarities, differences, and patterns.
4. Identifying Outliers: By examining the minimum and maximum values, analysts can identify potential outliers that may require further investigation.

Outlier:

In the context of the Five Number Summary, an outlier refers to a data point that significantly deviates from the rest of the dataset. Outliers can distort the distribution of data and affect the accuracy of statistical analysis. The Five Number Summary is particularly useful for identifying outliers because it includes the minimum and maximum values of the dataset. By examining these extreme values, analysts can quickly spot potential outliers that fall far beyond the interquartile range (IQR), which is the range between the first quartile (Q1) and the third quartile (Q3). Outliers may indicate measurement errors, sampling variability, or genuine anomalies in the data. Therefore, detecting and appropriately handling outliers is essential for ensuring the integrity and reliability of statistical analysis and decision-making processes.

Example Application:

1) suppose we have a random number dataset like this :

a=[10,12,10,9,24,56,43,12,89,89,190,87,-38,89,94,66,98,128,42,44,87,4,1000,1200]

Now,
we have to find out q1 which is 25 percentile of the data

q1=np.percentile(a,25)
print(q1)

// ouput is 12.0

we have to find out q3 which is 75 percentile of the data

q3=np.percentile(a,75)
print(q3)

// output is 89

and q2

q2=np.percentile(a,50)
print(q2)

// Output is 61.0

Now , we have to find out IQR:

IQR=(Q3-Q1)

IQR=(q3-q1)
print(IQR)

//Output is 78.25

Now we have to find out lower fence and higher fence which is 1.5 times q1 and 1.5 times q3

lower_fence=q1-(1.5*IQR)
higher_fence=q3+(1.5*IQR)
print(lower_fence,higher_fence)

//Output is -105.375 , 207.625

From here we can tell that the numbers which are not range between [-105.375 , 207.625] are outliers
so here we have outliers as we have 2 numbers which are out of range.

Let’s do a boxplot

From here it clearly visible outliers. which are placed above on higher fence.

2) Suppose we have a dataset representing the salaries of employees in a company. Using the Five Number Summary, we can quickly grasp essential insights:

import numpy as np
import seaborn as sns

np.random.seed(42)

salary = np.random.randint(10000, 10000000, size=1200)

sns.boxplot(salary)

From this summary, we can infer the salary distribution, assess the median income, and identify any outliers.

The Five Number Summary is a valuable tool in statistical analysis, providing a concise yet informative summary of numerical data. Its simplicity, robustness, and interpretability make it widely used across various fields, from finance and economics to healthcare and education. By understanding and utilizing the Five Number Summary, analysts can unlock valuable insights, make informed decisions, and communicate findings effectively.