This is a hot button issue for me as I'm currently fighting the same battle. (Only I'm fighting for myself, not on behalf of my child.)
From what I've experienced, people who don't have a food allergy or a child with a food allergy are never going to fully comprehend what we go through day in and day out.
My advice to you would be to stand up to your family and make them understand that you are his mother. You know what's best for him and you do not need any of their advice, especially when it comes to his food allergies! Remind them that food allergies are life threatening and as his mother you are doing what you need to do to keep him safe.
Don't be afraid to bring his own food that you have safely prepared for him in advance to family events or parties. Also, don't feel guilty if you choose to avoid certain family members or family events.

I found an article in a magazine about the death of a sportsman from peanut allergy. I left it for extended family to read and left them with the comment that for our son, peanut is only on of his allergens. (he had a very long list at that point.)

I am the child who's family thinks it's not that bad... my dad frequently asks if I'm over the peanut allergy hahah... it's only funny to me now because I know he's kidding, but we used to fight about it a lot until I gave him a nice reality check on the situation.

But dch05 is right, you have to stand up to your family. Typically people have no idea what it is like until they get a good talking to. I find that educating people is ideal. I tend to eat before family events or bring an already prepared plate and just heat it up while I'm there. No one really thinks twice about it anymore. One of my cousins actually developed allergies later in life so eating with her family is great because they get it and everyone is super careful!

I still argue w my fam. They just can't get it that I can't eat certain things mashed potatoes gravy ( usually milk n butter in them) brown sugar on sweet potatoes heck sweet potatoes any sugar

I avoid all relatives who think they know me better than I know myself. it's on them, not me.