Will switching Invisalign trays harm progress?

Stay with tray 13.  Wear two weeks and change.  There is no advantage to going back and forth.  Clean your trays with dish soap twice a day and you can put a teaspoon of bleach in a cup of water and soak it for 5 - 10 minutes (no more) to help the discoloration. Be sure you only drink water with them in and always rinse your mouth after eating and drinking (Brushing would be better)

I think you could have a problem soon with the tracking of your teeth based on your description above.  You only wore Tray 12 for 5 days, not even close to enough movement.  I would have went back to tray 11 and had the orthodontist order you a new 12.  Instead your moved to tray 13 and went back to tray 12.  This is something you should not be doing.  You are barely 1/4 of the way through treatment and at some point I think the tray will not fit right.  

Its ok to switch as long as they are fitting properly and you are following the instruction of your current dentist. If the discoloration of the trays has become a problem for you i would suggest you pick up some Polident from a local drug store which is a pill that you can put into a glass of water that will be able to clean your trays without damaging or altering the tray shape and size. Check out the link below for details on Polident product. Hope all turns out great with your Invisalign treatment.