I recommend wearing the binder for 6 weeks afterward, so as long as your plastic surgeon is okay with this you should be able to leave it off while starting out slowly lifting weights.
Surgeons will have different opinions and advise regarding the use of garments as well as when to return to exercise. it would be best to simply check in with your surgeon's office.