Ajeya Cotra is a researcher at METR, where she works on threat modeling for loss-of-control risks from advanced AI. Before that, she led the technical AI saf...

Ajeya Cotra – “This might be the clearest warning shot we ever get”

It is difficult to know what is true and what is deliberate marketing / misinformation / fraud intended to make LLMs seem more capable than they are. But if there is any truth to what is presented in this video, controlling LLMs to limit the means they might use to achieve an objective is not reliable, which is a significant risk.

Either way, I found the discussion in this video interesting. But I wonder what others know about how truthful this presentation is.

  • HakFoo@lemmy.sdf.org
    2·
    3 天前

    A legitimately interesting take.

    I can see two angles:

    1. AGI is implicitly expected to become ASI because many useful technologies have scaled to levels natural evolution didn’t.

    Our strongest weightlifters didn’t grow into thousand-horsepower cranes. Our quickest math savants are not cranking out a teraflop of FP32. But conversely, that’s why cranes and CPUs are economically interesting.

    1. Not really sure being able to carry on a plausible conversation on a “new subject” like the game example is much of an evidence of intelligence. ELIZA did it with a few kilobytes of RAM and some manually hardcoded rules. Having a huge set of training data just expands the opportunity for a plausible match.

    I think the lack of self-improvement and finite contexts are sort of damning. Outside of rare neurological disorders, I can’t make intelligent people forget how to ride a bike by reading the phone book to them.