Maybe. You should try saying up without the patting! Yeah, its eye contact when they aren't already on my head I honestly have a hard time keeping them off me. Sometimes I regret teaching that.I pat my arm and say "up". But I would have to guess that me patting the arm is more what tips her off.
That's cool that your do it with eye contact only!