If finasteride loses effectiveness would changing to dutasteride work or would it not? I know it's not FDA approved for hairloss

This needs an in-person evaluation. I suggest seeing a hair loss expert now to evaluate and go over your condition and provide you information about your best treatment options. There are great non-invasive options like prp/progesterone and/or hair transplantation. See an expert. Best, Dr. Emer

You are correct - while not formally FDA approved, dutasteride can be used off label in the treatment of male balding or 'androgenetic alopecia'. 
Finasateride is an inhibitor of the enzyme 5 alpha reductase type 2  and dutaseride is an inhibitor of both 5 alpha reductease type 1 and type 2. Dutasteride is more potent and leads to greater reductions of dihydrotestosterone (DHT). Dutasteride lowers serum DHT by up to 90% whereas finasteride lowers it by about 70 %. Side effects are also potentially greater with dutasteride than finasteride.
To answer your question, there are several points you may want to discuss with your healthcare provider. Patients who have a “partial” response to finasteride often wonder if they should switch to dutaseteride or add dutasteride to the finasteride they are already taking. Adding a very small dose of once weekly dutasteride can often be a good option. An Australian study in 2013 reported a male who was initially treated with finasteride for male pattern balding. Despite continuous treatment, after year 4 his hair density was not as good as previous years, and low-dose dutasteride at 0.5 mg once per week was added to the finasteride therapy. Interestingly, this resulted in a dramatic increase in his hair density, demonstrating that combined therapy with finasteride and dutasteride can improve hair density in patients already taking finasteride.
Another option is stopping finasteride altogether and starting dutasteride. Only you and your physician can determine if that is appropriate for you. In 2014, Jung and colleagues from South Korea studied 31 men with male balding who took dutasteride after finasteride did not help them.  Well over three quarters of these men improved their hair density by making the switch (17 improved slightly, 6 moderately, 1 markedly).
These studies suggested that dutasteride could be a good option for men if finasteride does not help completely.
Reference:
Jung et al. Effect of dutasteride 0.5 mg/d in men with androgenetic alopecia recalcitrant to finasteride. Int J Dermatol. 2014 Nov;53(11):1351-7
Boyapati A and Sinclair R. Combination therapy with finasteride and low-dose dutasteride in the treatment of androgenetic alopecia. Australasian J Dermatol 2013

The effectiveness of finasteride is about 70% at clocking 5-alpha reductase while the efffectiveness of dutasteride is about 85% making it a more powererful drug