Even analytical AI needs to be questioned and validated before use.
I wouldn’t trust a AI to ID mushrooms for consumption.
I forget the details, but there was a group training a diagnostic model (this was before “AI” became the popular term), and it was giving a lot of false positives. They eventually teased out that it was flagging low quality images because most of the unhealthy examples it was trained on came from poorer countries with less robust healthcare systems; hence the higher rates of the disease and lower quality images from older technology.
I’ve seen a similar thing where the machine learning model started associating rulers with cancer because the images it was fed with known cancer almost always also had a ruler to provide scale to measure the size of the tumor
It’s like these geoguesser not guessing the country by the plants and streets or houses, but by the camera angle and some imperfections only occuring in pictures taken in that country.
“When a measure becomes a target, it ceases to be a good measure”
analytical AI is great
generative AI is cancer
Even analytical AI needs to be questioned and validated before use.
I wouldn’t trust a AI to ID mushrooms for consumption.
I forget the details, but there was a group training a diagnostic model (this was before “AI” became the popular term), and it was giving a lot of false positives. They eventually teased out that it was flagging low quality images because most of the unhealthy examples it was trained on came from poorer countries with less robust healthcare systems; hence the higher rates of the disease and lower quality images from older technology.
I’ve seen a similar thing where the machine learning model started associating rulers with cancer because the images it was fed with known cancer almost always also had a ruler to provide scale to measure the size of the tumor
It’s like these geoguesser not guessing the country by the plants and streets or houses, but by the camera angle and some imperfections only occuring in pictures taken in that country.
“When a measure becomes a target, it ceases to be a good measure”
Palantir enters the chat.
Mass surveillance is bad regardless of whether or not AI is part of it.
Well there’s no “AI” so we don’t have to worry about it. Just a bunch of disgusting rich bros spying on us or worse.
This kind of wacky absolutism is part of the problem. Esp since “AI” doesn’t even exist.
https://www.downtoearth.org.in/governance/lavender-wheres-daddy-and-the-ethics-of-ai-driven-war