Finasteride side effects, are they going to go away?

Hello, two weeks ago I noticed I can't get a good erection and my semen is watery. I thought it's because of the finasteride. I never had a problem in the past three years. These past two weeks I started taking it at different times, sometimes even forgetting to take it. Could that be the cause? It's been 3 days since I stopped taking it but I'm not sure if I should stop since it really helped my hair. Please let me know If should get back on it or leave it forever.