If you get bitten by a racoons, are you going to say “not all racoons”, or are you going to get a rabies shot?

Get a shot, because a raccoon that bit you is overwhelmingly likely to be rabid.

“Not all men, but enough that you need to treat it like it’s all men” doesn’t quite have the same ring to it.

You don’t need to though. You choose to based on your own experiences and biases. When someone says it’s not all men they are basing that on their experiences and biases.

Neither “men” nor “women” are a monolith.