Hello, everyone! I hope everyone doing good. Sometimes it feels unreal that we used to not wear facial masks outside. There will be ways to overcome the current difficulties wisely. I find meditation helpful.
I. Introduction
Anyway, I’ve read a story written by one of our team members regarding CLT, and some undergraduate students might arise a question; what if the MGF does not exist?
Or is it a question that never occurred to your mind?
So, here we’re going to prove the ideas that might sound confronting to common senses to some students.
- Expectation might not exist for certain distributions
- Some distributions might not have moment generating function
CLT affirms that any distribution converges to a normal distribution. Then how can we prove the most important theorem in such a case when a distribution does not have MGF? In other words, proving the CLT using MGF method is limited in some sense.
As a solution to it in the next post, I will first introduce characteristic function (from now on abbreviated to CF), then by showing that the characteristic function of a distribution converges to that of normal distribution as number of samples goes to infinity, CLT can be proven.
II. Expectation
Expectation or average is a concept we’ve known for years. Even in elementary schools you get your average GPA (at least in our culture, which I used to loathe). For that reason, usually some newbies to Statistics take the idea of expectation for granted, and the sentence ‘some distributions do not have expectation’ might be mind blowing.
1. How have you been calculating the EXPECTATION?
Normally, we regard average and expectation equal. Back to the average GPA, if you receive 10, 8, 9, 7.5, 8.5 this semester, your average GPA is 8.6=(10+8+9+7.5+8.5)/5. Here 5 can be interpreted as a number of courses you’ve taken this year. However, from the perspective of expectation, the same equation can be interpreted as score multiplied by the probability of each course.
What we’ve been doing can be described in a more statistic context,
Here, the idea of ‘probability’ might be confusing. How about tossing a fair coin. You earn 10$ when it’s head, or lose 5$ when it’s tail. Then your expected value from tossing a coin would be 2.5=10*1/2+(-5)*1/2
2. Formal definition of Expectation?
In intermediate courses, you should have seen following equations for expectation;
However, there’s more to expectation. The actual definition of expectation is as following;
In graphs, the definition might be more straightforward.
However, this is still confusing. From now on, I’ll show how the real definition of expectation leads to our familiar equation.
However, I just want to clarify it is unnecessary to understand every details (changing the order of double integration). Although this is not that advanced mathematics, I just don’t want you to freak out. The most important thing is that the definition of expectation with which you’re familiar is just a simplification of its actual full definition.
3. Absolute Integrability Condition (AIC)
Then what is the condition that expectation exists? Having an expectation implies that the expectation has a finite value. We’ve had quite a detour, however, it is as simple as it sounds.
So, the condition for the existence of expectation or usually called Absolute Integrability Condition (AIC) is formally defined as following.
With some calculation, it can also be rewritten as following.
It is called absolute convergence condition, implying that the expected value of absolute value of x is finite.
4. Cauchy distribution
Cauchy distribution is well known for not having the expectation and the following moments. Since it is mathematically tedious to show that the cauchy distribution does not meet the AIC, I’ll skip the formal proof.
However, I think it is important to get the idea that there exist distributions that do not have expectation.
III. Conclusion
Knowing that expectation might not exist, it is equivalent to say that MGF does not exist in that situation. Then what? Are we gonna stay here disappointed? No! We have characteristic function that is guaranteed to exist in any distributions and we will be proving CLT using that concept. The proof of CLT from that perspective is actually show that CLT holds in any case even when the distribution does not have expected value.
Isn’t it surprising that the expectation that we all have regarded as given might not exist but CLT which we usually learn at the end of course and regarded tricky exists in any cases? For me, it did.
Anyway, I hope everyone has enjoyed reading the proof ;) and see you soon! Thank you.
Cecilia Kim.
