Hypothesis Testing — A Pragmatic View🎉

Ever wondered how vaccines for diseases like COVID-19, Polio, Small Pox etc., which were epidemic, went through clinical trails in humans of such high population and diversity even to the genome level🤔?? If so, you are not alone. 🙋‍♂️

Here comes Hypothesis Testing — A statistical method to test whether the sample data is good enough to represent the entire population data. Hypothesis is nothing but an assumption or a claim about an use-case at hand. e.g., Hypothesis can be like whether a new insulin drug has the efficacy of over 95% in humans battling with Diabetes. Hypothesis testing is used to either accept this claim or fail to accept the claim.

Now, we have two claims — Whether the drug’s efficacy is more than 95% or less than 95%. We call the first one as Null hypothesis (H0) and the later is called as Alternate hypothesis (Ha / H1).

Null & Alternate hypothesis

If you found that there is no evidence of drug’s efficacy over 95%, we say that the Null hypothesis (H0) is rejected and if efficacy is over 95%, we say Null hypothesis (H0) is accepted. We will not say alternate hypothesis is accepted. We will only say that either H0 is accepted or fail to be accepted.

Always, Null hypothesis is the status quo. It is our original claim, the alternate hypothesis is always challenging the original claim.

The null hypothesis always has the following signs: = OR ≤ OR ≥

The alternate hypothesis always has the following signs: ≠ OR > OR <

Non-directional Hypothesis

Let’s say a pharma company claims that their new kind of statin lowers cholesterol in 10 days. So, you got curious to know whether their claim is valid or not. So, You checked the cholesterol before taking the drug and only after 25 days, your cholesterol levels start to decrease. Now, you are sure that the drug is not efficient. But if the cholesterol levels decreased in 12 days, you will have an ambiguity whether the claim is true or false. The value that you think either in the lower or higher end of the claimed days is called as the critical value.

In the above diagram, you can see that if the average mean falls within the orange area, we can say that we accept the H0. If it falls with in the critical regions either on both sides in blue color, we reject the H0.

The above depicted diagram is called as Non-Directional Hypothesis. It means, if mean (average) 10 days is H0, then H1/Ha can be said as not 10 days. So, it can be less than 10 days or more than 10 days. This is an example of two-tailed test. We can see that we have two critical regions.

Directional Hypothesis

The above depicted diagram is called as Directional Hypothesis. It means that the calculated mean is less than 10 days. So, we are sure that the critical region is going to be present at only one end of the distribution. This type of hypothesis test is called as one-tailed test.

Critical Value Method:

A critical value is a threshold by which the rejection region is separated from Non-rejection region.

Critical Value Method

We have to select a quantity, alpha (α) called as significance level for the test. It represents the proportion of the sample mean lying in the critical region. For most use-cases, α is taken as 0.05 (or 5%). But for life critical domains like medicine, drug discovery etc., α can be 3% or even 1%.

Now, we have to find the right test for the hypothesis. For e.g., Z-score test. The formula for Z-score is as follows:

Z-score = ( x̅ — μ0 ) / (σ /√n)

x̅ = Sample mean

μ0 = Population mean

σ = Population standard deviation

n = Sample size

Let’s say the average height of men in India is 170cm. It’s population standard deviation is 10cm. we have collected sample height from 100 different men and their average is 175cm. Let’s formulate the hypothesis test using Z statistics. We can formulate the hypothesis like H0 is equal to 170 and H1 is not equal to 170.

We take the alpha to be 5% or 0.05. Since it is a two-tailed test, the critical region would be on both sides of the acceptance region which means 0.025. The value of the region till the UCV is 1–0.025 = 0.975. This will be the Zc (Z-Critical) value for the given significance level.

Next, we need to look into the Z-Table (https://www.z-table.com/) for the value 0.975 which comes out to be 1.96. Check the above link on how to calculate the Z value using the Z-Table.

Now, we have calculate the UCV and LCV (Upper Critical Value & Lower Critical Value).

Here, x̅ = 175, μ = 170, σ = 10, n = 100, Zc=1.96

σx = σ/√n = 10/sqrt(100) = 1

UCV = μ+(Zc * σx) = 170 + (1.96 * 1) = 171.96
LCV = μ-(Zc * σx) = 170 — (1.96 * 1) = 169.60

From the UCV, LCV and sample mean x̅ being 175, we can say that we reject the null hypothesis.

Summary — Critical Value Method:

1. Formalize the H0 and H1
2. Calculate Zc (Z critical value) based on the level of significance alpha (mostly 0.05 or may be lower). Note that if the problem is a two tailed test, we must use only 0.025 on either side of the mean distribution. The Z-value will be calculated using the Z — Table
3. Find out the lower and upper bound values and check with the sample mean whether the value lies in acceptance region or rejection region which the critical regions.
4. If the mean score lies in the acceptance region, we say H0 is accepted else we failed to accept the H0

P — Value Method:

This method is very important and widely used in the Industry. In the critical value method, we find the Z-score for the critical points while the same has to be calculated for sample mean in P-value based method. We also can say that P- value is the probability that the null hypothesis will not be rejected. If the p-value is higher, there are more chances that the null hypothesis won’t be rejected.

Formalize the H0 and H1/Ha as per the use case. Let’s say that, we want to find out whether the average price of a laptop in entire world is 5000 USD with standard deviation of 300. Studies have been done in 100 countries and the sample mean of it is 5200 USD. Here, H0 is average price is equal to 5000. H1 is not equal to 5000.

Since H1 ≠ 5000, we have to go for two-tailed test.

We have to find the z-score for the sample mean. x̅ = 5200, μ = 5000, σx = 300 / sqrt(100) → standard deviation/sqrt(sample size)

Z = ( x̅ -μ)/σx = (5200–5000)/300 = 0.66

p = 2 * (1 — Zscore (from z-table for the value of Z)) → (we use 2 to make sure that it is a two tailed test)

p = 2 * (1–0.7454)

p = 2 * (0.2546) = 0.5902

Now, we can conclude that P-value of 0.5902 is greater than the level of significance. So, we failed to reject the H0 (Null hypothesis).

Summary — P-Value Method:

1. Formalize the H0 and H1
2. Calculate Z for the sample mean in the given use case. Z-score = ( x̅ — μ0 ) / (σ /√n)
3. Find the corresponding value in the Z-Table for the value calculated in the step 2 (Z_final).
4. If the test is a two-tailed one, P = 2 * (1 — Z_final) else it will be P = (1 — Z_final)
5. Once you get the score from the step 4, check with the level of significance (α). If the p-value is lower than α, we reject the null hypothesis else we fail to reject the null hypothesis (H0)

Types of Errors:

There are two types of errors that we can make while performing the hypothesis testing.

Types of Hypothetical Testing

Type-I error (α) occurs when the probability of error occurring is 1% or 5% chance. This error is also called the level of significance of the hypothesis test. We shall also call this as a False Positive.

Type-II error (β) is more worse than (α). We shall also call this as a False Negative.

Decreasing either beta or alpha is based on the use-case at hand.

Hypothesis testing plays a key role in Clinical Trails, A/B testing, Strategic decision making, Manufacturing etc.,

Thanks for reading!