Statistical concepts

As the size of a system grows (linearly), the number of configurations grows exponentially.

The idea of information arises in the context of estimating a parameter \(θ\) from a sample \(Y\).

If \(Y\) contains a lot of information about \(θ\), then we have high confidence in the accuracy of \(\hat{θ}\). Conversely, if \(Y\) contains little information about \(θ\), we have lower confidence in the accuracy of \(\hat{θ}\).

Fisher information implements the idea of information using asymptotic sample properties.

The score is the derivative of the loglikelihood.

The score at a point \(\ell '(θ)\) tells us which direction to move in to maximize the loglikelihood. A positive score tells us that the MLE is to the right.

A big (positive) score means it is very likely that the MLE is to the right, while a very small score makes this unlikelier.

Given \(θ\), the score at that point \(\ell '(θ|Y)\) is a random variable (through a function of \(Y\)). We can take the variance of this score.

If the variance is low, the slope of \(\ell\) at \(θ\) changes little as \(Y\) is varied. If the variance is high, the slope at \(θ\) may vary wildly.

F

The score

The score at a point tells us which direction to move in to maximize the loglikelihood. A positive score tells us that the ML

If the score is

The Fisher information is the variance of score at the MLE \(\hat{θ}\).

\[\mathcal{I} θ = \operatorname{Var} \ell '(\hat{θ)}\]

As \(n\) grows, the likelihood of \(θ\) approaches a normal distribution.

Let

g

Let \(N_μ,σ\) be the asymptotic distribution of \(θ\).