Even the fears about AI is more Idiocracy than Terminator. Sure, this word-guessing program is going to guess enough words to suddenly gain consciousness…
consciousness is less relevant… a paperclip making ai doesn’t need consciousness to convert the world into paperclips
self improvement without a human in the loop is legitimately close, and once that happens it’s possible we lose control and alignment becomes incredibly difficult (alignment with human values is inefficient, so removing that barrier is low hanging fruit)
viruses also aren’t conscious or even definitively alive, but because they can effect the world on their own (which ai now can too to some degree), they can negatively impact us because they’re not aligned with the things we’d like
and on consciousness too, if you believe our brain is just chemical and electrical processes and there’s nothing more, we are just a big statistics machine, so there’s nothing technically stopping some program from being conscious, and in that case it’s kinda the “million monkeys at a million typewriters” situation… if AI speeds up AI development then things get exponential, and when you speed things up 1,000,000x who’s to say it doesn’t stumble onto a solution - it doesn’t even have to look anything like current models in that case… in 32 seconds it’ll have (theoretically; i’m not saying that’s possible or anything) completed a years worth of improvement, and models from a year ago were significantly less capable
there’s also emergent behaviour, and plenty of other weird things (heck LLMs in general are kinda emergent behavour… we didn’t think that simply increasing size would lead to the structure and behaviour that we see)
It’s really just Idiocracy.
Even the fears about AI is more Idiocracy than Terminator. Sure, this word-guessing program is going to guess enough words to suddenly gain consciousness…
consciousness is less relevant… a paperclip making ai doesn’t need consciousness to convert the world into paperclips
self improvement without a human in the loop is legitimately close, and once that happens it’s possible we lose control and alignment becomes incredibly difficult (alignment with human values is inefficient, so removing that barrier is low hanging fruit)
viruses also aren’t conscious or even definitively alive, but because they can effect the world on their own (which ai now can too to some degree), they can negatively impact us because they’re not aligned with the things we’d like
and on consciousness too, if you believe our brain is just chemical and electrical processes and there’s nothing more, we are just a big statistics machine, so there’s nothing technically stopping some program from being conscious, and in that case it’s kinda the “million monkeys at a million typewriters” situation… if AI speeds up AI development then things get exponential, and when you speed things up 1,000,000x who’s to say it doesn’t stumble onto a solution - it doesn’t even have to look anything like current models in that case… in 32 seconds it’ll have (theoretically; i’m not saying that’s possible or anything) completed a years worth of improvement, and models from a year ago were significantly less capable
there’s also emergent behaviour, and plenty of other weird things (heck LLMs in general are kinda emergent behavour… we didn’t think that simply increasing size would lead to the structure and behaviour that we see)