The main reasons for a surgeon operating in his office are cost, convenience and efficiency. The back-up services (cardiologists, anesthesiologists, other specialists, ICUs, special equipment etc.) are far greater in a hospital; so most people would consider that the hospital is indeed'safer.' The risk of infection in a hospital is not greater when we consider day surgery.
Unfortunately, hospitals are not geared to quick turnovers and the nurses and technicians are not selected by the surgeon. Operating is not so slick and can be frustrating when the staff is inexperienced.
Furthermore, hospitals are not particularly keen on doing cosmetic surgery, because they are paid much, much, more by insurance companies for reconstructive cases. They do cosmetic surgery as a favor to the Plastic Surgeons, who they hope will use them for insured services.
Nevertheless, if a patient has any serious medical condition, then the operation should be done in a hospital.