encrypt them with a password if you wish.

SSH keys without passphrases are just fancy credential files sitting in your .ssh/ directory, basically like writing your passwords on paper and leaving it in your desk drawer.