Can contact lens use tear the inside of your eye lid?

I've worn contacts for 4 years now and ever since a month ago when I went swimming and chlorine water got in my eyes, I've had problems wearing contacts. My eyes would become overly dry, especially under my eyelid and this would cause the contacts to slide up to the drier areas and get stuck there. And also when I remove my contacts, it feels as if I have an eyelash stuck under my eyelids and it's somewhat painful.