Thank you for your question. In your case, the minor break and the re-set should not prevent you from flying four days later, however, I suggest that you verify with the board certified facial ENT who reset your nose correctly.
It should not be a problem to fly 4 days after nasal surgery. However, this is something that you have to discuss with your surgeon. Good luck.
A minor nasal fracture can be reduced under local but a more major one would require anesthesia. Flying 4 days later should not be an issue depending on the nature of the fracture, check with your surgeon about travel restrictions.