shape analysis - Fourier descriptors: trying to classify objects

Describing my background: I have around 33 items labeled. For example, 3 pictures of the contour of a basil plant, 4 pictures of the contour of earphones, 7 of a mug, etcetera.

I'm trying to calculate N Fourier descriptors of a random picture in order to compare them with the other items, making the average of the distances between classes and saying: "This is a mug!" or "this is a basil plant!".

In my experience, I take a random picture of a MUG:

it looks to my MUGS that I've classified (because it gives me a distance close to 0):

but if I take a picture of the MUG without the handle:

the closest averages of the norm of distances to this MUG are of BASIL plants or EARPHONES (instead MUG, which is boundary closer):

indeed I have been realising that for every random picture I use to get a low value for earphones and basil. It seems that I have to normalize somewhere... but... what should I do ??

Do anybody could give me a clue?

Thank you very much in advance. Regards.

UPDATED In response to @tbirdal , I have to say that that's what I do:

vector vector;
if (dft_z.size() < 2)
    return vector; // handling errors
dft_z.at(0) = complex(0,0); // First DC to 0 (invariance on translation)

double si1 = abs(dft_z.at(1)); // Module of the a(1)
for(int i = 0; i < dft_z.size(); i++) {
    dft_z.at(i) = complex(dft_z.at(i).real() / si1, dft_z.at(i).imag() / si1); // dividing by the module
    vector.push_back(abs(dft_z.at(i))); // and putting into my vector the absolute value (to eliminate any phase)
}

HOW I'M GOING TO CONTINUE

I've seen that making the absolute value of every FD to remove its phase makes confusion on my shapes. In addiction, I realised that only simple forms and contours are well matched. For example basil was too complicated, but the mug is not.

Thanks to @tbirdal who gives me an idea.