Maximum Likelihood Estimation for Binomial Distribution Example

PDF for Binomial Distribution is

 Where y number of successes and w is the probability of success.

Suppose we have 10 Bernoulli trials of tossing a coin

The pdf In this case is 

The generated plot is as follows

Let us define a likelihood function for this simple binomial distribution .

L(w|y) = f(y|w) , suppose that the number of successes in an experiment is 7

So the likelihood function in that case is

The likelihood function is a function of the parameter whereas the pdf function is a function of the data observed , according to our example both functions are represented on different scales , the likelihood function is described on a parameter scale whereas the PDF function is described on a data scale , the likelihood function is a curvature function as follows .

 Steps to obtain a maximum likelihood function are as follows

1)      Obtaining the likelihood function as above for this specific binomial PDF.

2)      Take the logarithm of the likelihood because both are monotonically related to each other.

3)      Differentiate the log-likelihood function and let it equal zero. 

4)      To make sure that the likelihood function is a maximum not a minimum , we should calculate the second derivative and make sure that the results are all negative, since the maximum likelihood should be a convex or a peak.

So , in the above example ,  by taking the log of the likelihood function, we will have the following equation.

Next , we need to calculate the first derivative of the log likelihood function to yield the following

Now the MLE for the binomial distribution is obtained as above ,  now we are going to set that to  zero to solve for w which is the needed parameter for binomial distribution

7 – 10w / w(1-w) = 0

Solving this equation will give that w = 0.7 ………………………………………………….(MLE)

To make sure that, it is actually the maximum likelihood not the minimum one , we should take the second derivative of the log likelihood should be calculated and set to W(mle) to make sure it is negative as follows