5 days post-op breasts look much smaller than expected, should I be worried?

Thank you for sharing your question, and I am sorry that your current appearance has caused you concern.  After placement of implants the appearance of your breasts will change dramatically over the next several months, as the implants drop, and the overlying tissue stretches to accommodate their extra volume. The swelling from surgery is currently flattening your implants and holding them high on your chest.  As this swelling resolves, the volume within the implant will redistribute towards the lower portion of your breast achieving a more natural appearance, and filling your bra cup more.  I know waiting is difficult, but judge your results at 4-6 months after surgery as most of your breast changes should have occurred by then. Hope this helps.

Congratulations on having undergone the breast augmentation procedure.  Although online consultants will not be able to provide you with meaningful reassurance, it may be helpful for you to know (as you mentioned) that a significant percentage of patients at your stage of recovery will feel that they are too big or (more commonly) too small.
I routinely ask my patients to wait at least 3-6 months before evaluating the end results of the breast augmentation surgery. This waiting time allows patients to (usually) physically and psychologically adapt to the new body image. 
From the physiological standpoint, it takes many months ( and even up to one year) to see the final results of surgery. factors such as swelling, tissue contraction, skin redraping, implant shifting etc. come into play. You may be pleasantly surprised wants to be "drop and fluff" phenomenon has occurred. You may also be happier once the swelling AROUND the breast implants dissipates and the implants' shape and projection become more visible (projecting). 
As breast implants "settle", they tend to round out the lower breast poles and the nipple/areola complexes shift upwards ("drop and fluff"). In my opinion/experience, the "drop and fluff” phenomenon does hold some validity; as breast implants “settle”, many patients feel that their breasts actually look larger.
From the psychological standpoint, patients undergo an “adaptation period" during which time they get used to the changes that the surgical procedure has brought about. Severe emotional swings can also occur after any type of surgical procedure.
 Best wishes for an outcome that you will be very pleased with long-term.