How does one know if turbinates are enlarged because of allergies or born with them too big?

i have breathing issues and had 3 rhinoplastys but was told inside everything looks good by ents and plastic surgeons. I wanted to know how does one know if turbinates are enlarged because of allergies or born with them too big. if you misdiagnose and go with surgery doesnt that mean when the allergies go down the turbinates will actually shrink below the normal size? IDK what to do only afrin works so far. I just dont want permanent damage via surgery. Should i try allergist instead of ent

No doctor answers yet