Estimate the Discrete Fourier Series of a Signal with Missing Samples

Assuming we have a discrete signal $ { \left\{ x \left[ n \right] \right\}}_{n = 1}^{N} $.
Which has a Discrete Fourier Series.

Now, assume I'd like to estimate its Discrete Fourier Series coefficient yet some samples of $ x [ n] $ are missing (The indices are known).

How could that be done efficiently without computing the Pseudo Inverse of the adapting Fourier Series matrix?

Thank You.