My brackets keep breaking and my orthodontist is threatening to cancel the treatment?

My brackets keep coming of. I do not think it has anything to do with what i am eating as i always eat soft food and make sure any hard food i eat i cut into small pieces. It has got nothing to do the way i brush (this is what the orthodontist told me). I do not chew gum, my finger nails or pens. The orthodontist is also threatening to cancel my treatment if they come off again. I can barely eat comfortable. it feel like every meal is like walking on eggshells. What should i do?