Why do my Invisalign retainers hurt? Yesterday was my last full day wearing Invisalign.

I wore them that night & removed them in the morning since my dentist said my last tray is my retainer. However once I put them on before bed the next night they felt super tight like if I had put on a new tray. Is it possible my teeth moved in the span of 15 hours? I’m afraid to take them off during the day as I’m supposed to as I’m scared my teeth will shift during the day. Is it possible for my teeth to drastically shift back during the day?