Should I be really worried?

First and foremost, you should see an obstetrician/gynecologist for
determining the effects of surgery and medications on your 4 week old baby. This
should be your primary concern. Evidence suggests that certain drugs can have
negative effects on the fetus during the 15th and 56th days of gestation (i.e.,
8 weeks of the 1st trimester). Fortunately there is no known effect on the
fetus from most of the drugs commonly used in the breast augmentation procedure,
especially when not used for the long-term during the first trimester. If your
pregnancy had been known, you would have not been cleared for an elective
surgical procedure. The breast implants themselves will have no impact on your
pregnancy.

Secondly, the pregnancy and its related hormonal changes will gradually
cause your breasts to swell with milk, causing stretching of your skin.

After delivery and nursing of your baby, you may notice changes in
shape/size of your breasts (e.g., they may sag/droop requiring a breast lift in
the future or may lose some breast tissue).

If you plan to have a breast lift, you may want to have it after your final
pregnancy to avoid similar problems.

A complete evaluation from a board certified plastic
surgeon including review of your prior records, medical history, and review of
your surgical notes would be important to guide you about next steps.  

For now don’t worry about anything but your pregnancy.
Your body needs time to recover, and generally speaking, final results of
breast augmentation come about after 3-6 months.

Take
care.