Why do asteroids NEVER HIT US!?

Ok i know asteroids hit us. But have any of you noticed biggens actually narrowly miss us every year and populated areas NEVER get hit. I mean its described as luck but last time i checked luck doesnt last that long. We get mive near hits every year and they never hit. Am i the only one that thinks something fishy is going on here? Oh and did you know theyve got some truly bad idea's on asteroid defence!