Who actually decides what judges get what category?
 

Probably a random team of experts and psychologists who've watched the show... but I dunno.

nah I bet Simon does, just acts like he doesn't!

Not sure but it's not done fairly. Cheryl's got a strong catergory three years in a row now, the producers obviously know giving her a good catergory will mean more airtime for her.

The producers. As Callum said though, it's not done fairly at all.

Simon calls the shots oon everything else and he knows how to create drama and headlines so I'm guessing he chooses which judges gets each category.

They said on Xtra factor that Bill Oddie usually phones the decision in. Has done for years

The producers, but I think Simon is part of it & just acts like he doesn't know.

It doesn't make sense because X Factor is HIS show, he calls the shots... it must be what I suggested in the OP.