How bad will scarring from lipoma removal be?

The decision to remove the sutures after lipoma removal depend on the type of sutures and the technique.  Trust your surgeon with his advice. 

The scarring after a lipoma removal is dependent on the extent of the surgery that he received and how well your body heals scars. If your body is predisposed for thick scars, it is very likely that you will have a thick scar at this site as well. If the lipoma was large and required the rearrangement of tissue or the tissue was closed under tension, you are at risk for having a thick more prominent scar here as well.
 

The type of scar which is left behind from your lipoma excision depends largely upon the type of closure your surgeon performed. If he used a running "subcuticular" stitch, which is a continuous suture that runs beneath the surface of the skin, then the scar should not be affected adversely by removing it at two weeks.
If he used an "interupted" closure, or multiple sutures which are tied above the skin surface, then there is a chance that the scar could develop a "railroad track" appearance after two weeks. This is caused by scarring along each of the suture holes left behind by the stitch. I would recommend discussing this with your surgeon if you have not already done so.