audio - inverse FFT when only a few time points are needed

I am developing C++ code to do some modeling that will take as input a parametric model of the complex frequency response (amplitude, phase) and produce as output an impulse response, e.g. using the iFFT. I would like to compare this to a "windowed" impulse response that I have measured for a loudspeaker in a room. The measured impulse can be characterized by an initial portion that corresponds to sound traveling directly through the air to the microphone that lasts a few millisecond, after which sounds that reflect off of the surfaces in the room (floor, walls, etc.) contaminates the signal. The uncontaminated portion is only 3% of the total measured impulse. For instance if I record 16k points at 96kHz, its only the first 470 points of the impulse. I want to compare the impulse response that the iFFT generates from the frequency response model to ONLY the uncontaminated portion of the measured impulse.

I could calculate the entire impulse response from the frequency response and then just throw away 97% of the result but this seems to be very inefficient. The iFFT will be calculated many, many times (thousands probably) while my model is being optimized so I want to make sure that I can make it as efficient as possible. At this point my only option seems to be using FFTW and then just throwing away the data that is not needed (for lack of a better idea).

Is there a fast way to calculate the inverse FFT only for those 470 time points of interest, e.g. not for the entire time span that the FFT can access? I am not intimately familiar with the computation of the FFT and iFFT, so I don't have insight on the answer to this question.

Answer

If your inverse FFT is producing a time series that is N times longer than you need, downsample your frequency domain signal by N and then take the IFFT. Try it!

Edit: As has been pointed out, this method will produce undesired time aliasing unless the content after the window of interest is very small or non-existent. This method may still be favorable in cases where the system response decays exponentially. However, if your window ends before the response has decayed by say, 20 or 30 dB, you should consider another method that will be free of aliasing. Either that, or scale your window down only to the point where you are not negatively affected by aliasing, and then throw out any un-needed samples from there.