Jews weren't openly walking around Germany without a care in the world at that stage. They were already being attacked, ghetto-ised, murdered, and having their possessions stolen away from them and given to pure blooded Germans.

You would know if they were Jewish, as they had to wear a badge highlighting that very fact.