Can androgenic alopecia affect neck hair/the nape of the neck in men?

This needs an in-person evaluation. I suggest seeing a hair loss expert now to evaluate and go over your condition and provide you information about your best treatment options. There are great non-invasive options like prp/progesterone and/or hair transplantation. See an expert. Best, Dr. Emer

I'm sure you have learned that hairs grow in cycles through anagen, catagen, telogen and back to anagen again.  Every hair on our head (most people average 100,000 to 120,000 head hairs) under normal circumstances will grow on average for 6-7 years and then fall out and be replaced by a new one. This is why you never see anyone with hair that has grown below their knees (even if they have never cut it).  As humans we don't shed like dogs, instead the hairs fall spread out randomly across the scalp as they reach the time to fall and be replaced.   
This means that it is normal to lose 100-150 hairs every day spread around the scalp as part of this growth and replacement cycle.
The other thing that could be contributing here is that as we mature, men will have a mature hairline.  This means that some of the baby hairs around our face and lower neckline will leave.  For most people this happens some time between about 17 and 28 years of age.
How to tell if your hair loss is normal?  Find a doctor who can perform a microscopic examination by magnifying your scalp 1000x.  At that level an expert hair Doctor will be able to tell the difference between hairs that are maturing and hairs that are miniaturizing due to androgenetic alopecia. 

Without any changes going on i the crown or frontal hairline, it’s unlikely to be AGA. Obesity, excessive fat in the area, and various scarring alopecias can cause loss there. See an expert if worried.

In some individuals, the neck hair is genetically programmed to fall out early. Rarely have I seen this in a 20 year old, but not uncommon in a man over 35. A few hairs may not mean that you are losing neck hair but just normally cycling hairs