I actually like this head-canon and I'm going with it. Countless dark wizards have sent themselves back in time to kill Harry Potter, Terminator style, but NO ONE KNOWS ABOUT IT because there's a taskforce of magical future commandos that swats them like flies as soon as they try, using their superior future-magic.