Wear you wearing it front facing or backwards?
I only ever wear caps backwards and as far as I can remember I've never been asked to take it off in a restaurant (although I don't know if I've actually worn one in a restaurant tbf). I'd wear it in college classrooms too and teachers would never take issues with it
