Should I Go for a Smaller Chin Implant?

It is impossible to tell whether a chin implant should be removed and a new one re-implanted if it has only been three weeks. One needs to wait two to three months to make sure all the swelling has subsided to make sure that the implant is or is not the appropriate implant for the patient. Numbness usually subsides or disappears after two to three months as well. The implant should not be felt sticking inside of the mouth, and that could potentially be a slippage of the implant. The implant itself should not be painful after the first couple of weeks.

Implant size and numbness are two different questions. At this point given that you have had two operations in a relatively short period of time I would wait several months (at least 2-3) before doing anything with the implant unless you can definitively tell that the implant is resting on the mental nerve.

It sounds like the implant location was corrected at the second operation. You mention that it may be too high on the left side. Check with your surgeon and if this is the case it will need to be corrected with another procedure. The size of the implant should be determined by the amount of correction you want to achieve with the procedure.
Chaging implant size at this time would not bring back the sensation to your lip and chin any faster. Your smile will also get better as more time and healing takes place. All of this is due to the the nerve that was probably stretched during the surgery and the muscles that got swollen. As more time goes by the muscles will begin to work better and the nerves will start to function. More time for healing will be you best stategy for the abnormal nerve sensation and muscle movements.