I’d personally put them into a Saw trap, where they either can just give up on AI to the point all their data must be wiped, or they can do various “challenges” to save trillions of humans in the future.

Hello Mr. Altman, let’s play a game! You’re building an artificial general intelligence due to effective alturism reasons, don’t you? You think great sacrifices must be done today, to save trillions of potential humans in the future.

The room is slowly getting filled with water. Both your legs and one of your arms is chained to a machine. There’s a button on the machine, that you can press to release the chains, but it will also shut down and erase all LLM developments. Alternatively there’s a saw and some cable ties in your arms reach, so you can do some sacrifice yourself too to save trillions of potential humans in the future. You also can choose to just drown in the water, however doing so won’t guarantee that I or anyone else will let the AI development continue on its current terms.