Except you’re still trusting a lot of people and systems there. Those that programmed, compiled and/or packaged your software in use (be it e.g. the cryptographic libraries themselves, the OS, random user space applications you are running that might be able to access your mail some way or another…), the hardware you use, the software, hardware and OpSec of the recipient…

The amount of people who have actually the resources, time and knowledge to eliminate all these points (i.e. reviewing the entire source code of all the software you use, and all the diffs of every new release you use, somehow check all the firmware blobs for your hardware or manage to get a fully de-blobbed system running and connected to the internet, and otherwise making sure your keyboard doesn’t sent a copy of every keystroke to “the enemy”, …) is very low. And the amount of people who actually do it might be zero? Not even a person in the NSA will have done all of this themself. They’re trusting some coworkers for some of these parts…