I assume she was simply unaware of who Bebe was and saw an opportunity to solidify her position with Kai and make him believe that she couldn't be the mole. I assume if she did know who Bebe was then she would have still killed her since she put the cult in motion.
Ally wants to be the one to destroy the cult. I don't think she would have sat back and let someone else take that away from her.