noise - Where does the delta function come from if we derive autocorrelation directly?

I am reading a book "Creating Noise" written by Hollos & Hollos and have a question about the autocorrelation function of the Gaussain white noise when reading the following passage:

From the theorem that the autocorrelation and psd are Fourier transform pair and the fact that psd of Gaussian white noise is $\sigma^2$, it is obvious that the autocorrelation of Gaussian white noise has a delta function $\delta(\tau)$ as formulated in (6). So at $\tau=0$, what we have is a pulse of inifinite amplitude. But if we derive the autocorrelation directly, as does the book in the passage, we only have $R(0)=\sigma^2$ which is a finite number, instead of an infinite pulse. I believe the autocorrelation result having a delata function is correct, but the direct derivation of it looks correct too. Being not able to find out where the mistake is, I am curious where the delta function $\delta(\tau)$ comes from if we derive the autocorrelation $R(\tau)$ directly from its definition. I wish you can give me some help with it. Thanks a lot.

Answer

It turns out that the book is true only for discrete-time white noise. For continuous-time white noise, that's another story.

Let me start from a brief explanation in discrete-time white noise. In this case, at each isolated time instance, the value of the stochastic process can be regarded safely as a random variable. Note that for discrete signals, the delta function $\delta(0)=1$, not infinity, and its (discrete-time) Fourier transform is constant 1 as we all know.

Now we come to continuous-time white noise. The key here is that, at any isolated time instance, the value of the stochastic process CAN NOT be regarded as a random variable. As a result, it is incorrect to copy concepts in elementary probability theory and say, as in the book, that $R(0)=E(X^2)=\sigma^2$ where $\sigma^2$ is assumed to be a finite real number, e.g., variance of a Gaussian distribution, because there is no random variable and its pdf at all! If we must denote $\sigma^2=E(X^2)$ for simplicity, we must have in mind that this $\sigma^2$ is actually an inifinite value according to the definition of continuous-time white noise, that is, it itself contains delta function $\delta(t)$. I got to this point only after reading wikipedia for white noise. I suggest you take a careful read of the section on continuous-time white noise too, then you'll understand what I say above. The key to my question, to repeat, is that at isolated time instance, the value of the continuous-time white noise is not a random variable any more, so we can't (mistakenly) apply our knowledge of elementary probability theory, as did the author of the book.