Should I Have Painful Lipoma Under Breast Removed?

I have a lipoma under my left breast, its painful to touch. Should I have it removed? I thought I hurt my ribs, but after xray and untrasound Its lipoma. It hurts if I dont wear bras and to touch it. I get a pain when I move in different direction quickly, like a sharp stabbing pain in my ribs and have notice that its got bigger.