Maybe we could put back the four who were voted out.
Then the new four who were voted in, can then go out, only to go into a secret house. And then the original four who were voted out, and were then put back in, can be voted out again, and put into the secret house along with the four who were voted in after the original four were voted out and in again.