What is this pea sized lump on the side of my bridge after rhinoplasty? Should I be concerned?

You have to give yourself some more healing time.  Your nose and surrounding areas are going to be swollen for a while and you should be consulting the surgeon that performed the procedure. However, give yourself some time before going back in so some of it can subside. It can take up to a year for final results.
Sincerely,
Philip J. Miller, MD, FACS Double Board Certified Facial Plastic Surgeon

Swelling or graft or scar, it will certainly improve over time. Wait at least 6 weeks and then ask again your surgeon. A second opinion is really something not necessary at this stage. Relax and do not look at your nose too much for the next couple of weeks, though I know it is difficult not to. 

It's very early in your recovery period. There can be small irregularities that are visible early but resolve with time. Small problems like this can also be easily improved with a touch-up procedure in the office under local anesthesia down the road. Try not worry right now.