Fair point I suppose. He messes up whether the h should be pronounced or not half of the time anyway, and is not very good at pronouncing it even when he decides to try.
it would probably be safer for him to just stick to an, even though that would just lead to him keep saying "I'm an angry man" rather than "I'm a hungry man".