Can I drink alcohol with Invisalign on?

I often go out and stay 3-4 hours in the pub or club...I drink alcohol with invisalign on because I don't want to lose 4 hours without invisalign on. Can alcohol like vodka, wine or whisky damage the invisalign (dissolve or change the shape) or the only issue is decay and sugar? . I do not worry about this because my cleaning habit is very good).