Its important because woman use eye contact to signal attraction, if you avoid it not only do you prevent them from signaling to you, it also makes them feel invisible and unimportant to you. Think of it like you're walking into your boss's office, and the whole while you're speaking to him hes responding but still primarily focused on what he was already doing, you feel secondary.