The problem is that we don’t really feel like we are talking to anyone when we are davening. Rav Pinkus quips that a person only needs a siddur when he’s talking to the wall!