I think that it is a combination of a few things:
1. the people they happened to pick,
2. the edit,
3. the perspective of the viewer.
And I think 3 is the main point there. I have seen Sivash stressing out plenty... and Marcus (a bit) and Freddie (quite a lot) - maybe the male viewers react more to the female HMs stress because they understand it less. Maybe guys recognise and understand other men's stress better and it is therefore less stressful for them to watch it. Also, it seems that the women are more inclined to off load their stress (talking, arguing, bitching. etc.) and the men internalise their a bit more (duvet diving, long bouts of not talking aka sulking).
I also see plenty of stress being externalised by Rodrigo, Charlie and David - but not much coming from Hira or Sophie.
IOW: I think the OP may well be full of sh1t (but I mean that in a nice way
)