I had a rhinoplasty/septoplasty. Internal splints were removed after 4 days when I complained. Was this too early?

The original plan was to remove the external cast and nasal splints 7 days later however, I called my surgeon 4 full days after surgery to tell them I was still almost completely blocked from breathing through my nose and wanted to know if that was normal and they said to come in. They suggested removing the internal splints early to relieve some breathing issues. I asked if this would cause any complications and they said in most cases, no. So I agreed. I am concerned I should have left them in