Thursday, April 18, 2019

ica - Why non gaussian variables are independent


I have read a lot about ICA. I think I could not find the answer that why non-Gaussian Variables are independent. What I understand, Central Limit Theorem states that



distribution of sum of independent variable tends toward more Gaussian than its original random variables.



$ s_i $ are the number of original independent sources in ICA; whereas ICA model is $x=As$. So we can define $y=w^Ts$. The main goal is to find the unmixing matrix $w$ that maximize the non-Gausaanity. So my question is what is non-Gaussanity here and why its necessary to maximize it to extract the original sources.



Any Enlightenment Please!



Answer



The model ICA uses says that there exist some unknown, statistically independent sources, $s_i$ that are non-normally distributed (their distributions are something other than Gaussian):


$$ s_i \sim S(\mu_{s_i}, \sigma^2_{s_i}) $$


where $S$ is some (possibly) known but non-Gaussian distribution with mean $\mu_{s_i}$ and variance $\sigma^2_{s_i}$.


Then it is assumed that what you can actually measure is the addition of these: $$ \mathbf{x} = \mathbf{A}\mathbf{s} $$ where $\mathbf{s}$ is the vector of the $s_i, i=1,\ldots,N$, $\mathbf{A}$ is an $N\times N$ matrix (usually assumed to be invertible) and $\mathbf{x}$ is the vector of the actual measurements.


Because the $x_i$ will just be weighted sums of the $s_i$: $$ x_i = a_{i1} s_1 + a_{i2} s_2 + \cdots + a_{iN} s_N $$ the central limit theorem says that the distribution of $x_i$ will be closer to Gaussian than the distribution of the $s_i$ (provided certain non-restrictive conditions on the true distribution of the $s_i$ are met).


Then all ICA really tries to do is to find a $\mathbf{W} = \mathbf{A}^{-1}$ the inverse of $\mathbf{A}$.


There are any number of measures for "non-Gaussianity". One of the simplest (?) is to use kurtosis as the measure of how far the sample of a random variable is from Gaussianity.


So, to attempt to answer:




So my question is what is non-Gaussianity here and why its necessary to maximize it to extract the original sources.



There are several different ways one could measure non-Gaussianity. For example, kurtosis can be used as it is known that the kurtosis of a Gaussian is 3. Any distribution with a kurtosis different from 3 is therefore "non-Gaussian" to some extent.


The reason we want non-Gaussianity is because we assumed that the original sources are non-Gaussian.


No comments:

Post a Comment

digital communications - Understanding the Matched Filter

I have a question about matched filtering. Does the matched filter maximise the SNR at the moment of decision only? As far as I understand, ...