My ear is sore, swollen, has hard dead or dried skin that's flakey, and I do wrestle without headgear in practice.
You can be developing Cauliflower ears. Dry flakey skin is not associated with that deformity. If your ears are sore, you might not be washing your ears and removing the outer layer of skin. It is not possible to tell if you are developing a cauliflower problem without photos.