Is touching the recipient area on 4th day after FUE a cause for concern?

I don't recommend touching your surgery area for about a week after surgery. Touching your head afterwards won't cause permanent damage. But also, you should get contact your doctor first and ask for information.

There should be no issue with touching your head or hair after the surgery just as long as you are careful.  If you are concerned, follow up with your doctor. 

If there's no visible sign that anything is wrong, then I doubt you could have messed up any of the grafts. 4 days is a decent amount of time so the grafts shouldn't be overly sensitive to light contact like you described, and at this point it would take a little effort to really cause them damage.

If the touch was a gentle touch and there wasn't any bleeding then I suspect everything is fine.  

If there is no bleeding or no hair folucule is come out on the fourth day,  there should be no  worry regarding the root damage or hair growth of new hairs.

In my practice I advice my patient to not touch the recipient area for ten days because  the grafts became secure within 7-10 days . In your case if there is no bleeding , nothing to worry . All the best 

touching the recipient area after a hair transplant is never a good idea as germs from your hand can possibly bring infection to your recipient area

you can start to touch the recipient Area the day after the procedure. You just need to be gentle with that area for the first ten days, once grafts are settled you can return to normal routine.