No they shouldn't, unless the pool operator demands swim caps it's your choice. If your not doing swimming lessons in a class or are in a water polo team or are doing swimming club race training there is very little point in men wearing swimming caps.
In the same way that non competitive male swimmers don't bother shaving their bodies.

Even the French, who have banned shorts in their pools don't demand that you wear a swim cap.

In all seriousness, I've been a good swimmer since my earliest memory. I don't know if a Swim cap has a speed enhancing less drag for competition, or is it just about protecting hair & other swimmers have go go through long hair that's not there all the time. Serious question. I was even on the West Hollywood community swim "team". Never bothered to ever ask. Thanks :)