I would look back to old photos and see if there was any asymmetry when you were younger. As we age subtle asymmetry can become more apparent. If you don't feel confident with your doctor then I would get another opinion. Often these asymmetries can be corrected with filler or surgery. But you were right to rule out anything medically problematic with a CT and full exam.
Many factors can cause one eye to appear sunken - or the other eye to look proptotic (bulging). In some patients it is difficult to determine by history alone, which is the abnormal side. Just a few of the many problems causing proptosis or enophthalmos: thyroid disease (with or without normal blood test), trauma (recent or remote), sinus disease, mass effect, etc. Measurements on examination can determine if your problem is changing (serial orbital exam) - so it might be beneficial to go back to the same Oculoplastic surgeon for repeat measurements and another checkup to see if anything has changed since the last visit. If you are not comfortable with that doctor, then you could obtain a second opinion from another Oculoplastic surgeon. Be sure to take any documentation that you have - photos from the past, CT scans, examination records - and prepare your questions in advance so that you get the answers regarding diagnosis and options for treatment. Best wishes on getting the answers.
No photos are supplied. Since you have already started an evaluation of the problem, you should seek a second opinion if you do not have confidence in the first surgeon that you saw. Please take your studies with you when you go for the second opinion.