• 3 Posts
  • 21 Comments
Joined 3 months ago
cake
Cake day: June 5th, 2026

help-circle




  • Yes. There have been plenty of cases of people prompting an LLM to write “in the style of” of a specific author. There are also AI-generated books being sold on Amazon under real authors’ names, AI-generated books flooding self-publishing spaces, and plenty of other issues. Sometimes that stuff gets taken down, but most of it continues and makes it harder for real writers to gain visibility. On the technical side of things, if you want to run a “successful” blog with useful information these days, good luck. Google doesn’t send nearly as many people to actual websites today, mostly because of AI overviews, and a lot of AI-powered SEO content is outranking the human writing in SERPs. Sites like Medium are also mostly AI slop now. Human writers are getting crowded out.

    Much worse, a lot of real writers have been falsely accused of using AI. This is more of an issue if you blog or write articles, but it’s happened in fiction too. People will run your work through “AI detectors” even though none of them are that reliable. I’ve run into this issue before with my past writing (some things I wrote were “detected” as 20-60% AI-generated). AI is trained off of human writing patterns, so inevitably, people who have had their work over-represented in training data end up having AI-like writing patterns when really it’s the AI that sounds like them.

    I hate what AI has done to writing communities, too. People accuse writers of using AI on social media all the time. The site where I posted fiction as a teen has a lot of unlabeled AI-generated stories these days, and it’s known that a lot of older stories were scraped, so things are looking worse for the next generation of writers.

    I’ve also felt a huge decline in respect for writers of all kinds since 2022. A lot of people don’t read that closely and genuinely believe a chatbot’s output is equivalent, or at least good enough. And some of the people who do read closely are more suspicious. Like I said, there are plenty of false accusations of AI use going around. It’s just a much more antagonistic environment to write in.

    And those are just some of my issues with AI. I really, really hate being forced to contribute to the technology causing all this bullshit.



  • I don’t agree. Yes, under our current rules, posting things in public means people can do things you might not like with your work. And technically you agreed to Instagram’s BS when you made an account. But for all practical purposes, the rules changed after the fact for most people.

    Someone who made their Facebook account in 2009 couldn’t reasonably have guessed that their posts might be used to train misinformation machines that would be forced into everything in 2026. Most people could have conceived of strangers looking at their photos and maybe even photoshopping them, but could never have conceived of gen AI in its current form until it arrived. Bloggers could have conceived of people plagiarizing them or taking their words out of context, but not of having their own past work used to feed tools designed to replace most of their future work opportunities while also spreading hallucinations. And some people might have made very different choices if they’d known, myself included. No one gave informed consent for all that data to be used for LLM training, because no one could have imagined what they were consenting to. Even now that we can, we’re being held hostage in a sense. You can’t meaningfully opt out of AI scraping without giving up participation in society, not when leaked private data and copyrighted works end up being used for AI training too.

    As for looking for alternatives, it’s true that you can leave Instagram and use a different search engine. But there are limits. Most people are stuck with whatever software their job or school forces on them, even if they use alternatives in their personal lives. I can’t just tell my boss I’ll no longer be using Google collaboration tools or Microsoft products. I can recommend alternatives, but I don’t have the power to make the entire organization switch, especially when said boss actually likes AI. And almost every other office job in my country will likely require using the same software.

    You’re right that we should never have seen big tech as a benevolent public square. You’re right that from a legal perspective, this person has no grounds for complaining about Instagram. But from a human perspective, they do. I think it’s quite reasonable to be upset that anyone who ever posted anything publicly for other humans to appreciate has been effectively forced to contribute to gen AI training. I think it’s reasonable to be frustrated that many of the common collaboration tools most of us are forced to use (for now, at least) have had AI stuffed into them. In a better society, we’d be able to share things publicly with other humans while opting out of tech we find harmful.




  • Some search engine results are crap, yes. But even the worse search engine (minus AI overviews or whatever) gives you a setup where: (1) each piece of information has an actual source, (2) you know who the source is, and (3) you can make decisions about which information to trust accordingly. If a popular search engine puts a sketchy company’s blog at the top of the SERP, you can actually see that and choose to look father down for something more reliable. Also, if your current search engine keeps failing you, try a different one! There are many options out there that you might find more helpful than today’s Google or Bing or DuckDuckGo.

    Good luck with things, truly. I think you’re going to get hurt eventually if you use LLMs for medical advice, and I genuinely don’t want that for you or anyone. Hopefully this conversation has at least made you a little more skeptical about using LLMs in certain ways.


  • You can teach people how to use it all day long, but most still won’t use it well consistently in real life. Humans instinctively trust confident-sounding language. Most of us are also way worse at fact checking than we think. LLMs play right into humanity’s flaws, and they’re built to maximize engagement rather than help us avoid going past the tools’ limits. I don’t trust LLMs to be accurate about anything, but I still caught myself reading and failing to check Google AI overviews sometimes before I changed my default search engine to a no-AI alternative. It’s just instinctual when an “answer” is right there at the top of the page in response to your search question.

    Sure, you can use an LLM to help you find sources, but at that point you might as well just use a search engine. There are search engines that provide links with actual text from each site rather than potentially hallucinated summaries.


  • Limiting medical knowledge from reliable information resources would be terrible. I’d say that limiting the spread of plausible-sounding medical misinformation generated by a fancy autosuggest tool is good, though.

    AI chatbots don’t actually “know” anything. They just generate the most statistically probable appearance of a response to each prompt based on their training data. They don’t tell you when something wasn’t in their training data or when there’s limited research on a subject, just give you the most statistically probable response like always, which leads to a “hallucination” that sounds true but isn’t. A tool that’s trained on MCAT testbanks and “aces” the MCAT won’t necessarily give reliable medical advice to real patient questions. Even if you gave an AI chatbot all the accurate training data in the world, it would still hallucinate information occasionally, and it wouldn’t give you any indication of when it did so. If your doctor screws you over, they can lose their license, but if ChatGPT screws you over, there’s currently very little to stop OpenAI from screwing over more people. OpenAI can just stand behind the disclaimer that says ChatGPT can make mistakes, never mind that users are treating it like a licensed medical professional. There are AI tools that are useful in medical research, but they’re a VERY different kind of AI from the AI chatbots this bill is concerned with.

    If you want medical knowledge, there are tons of medical journals, news sources, blogs, and podcasts from doctors/scientists who don’t hallucinate BS when they don’t know something.






  • That looks far better than the mainstream AI tools, but I don’t think respecting opt-outs is quite enough. It would be so much better if it were built from solely opt-in training data. As far as I can tell, it’s not attempting to tackle the hallucinations or environmental impact issues. Still, it would be a major change for the better if ChatGPT users switched to something like that.


  • You hit the nail on the head. They produce output that mimics the appearance of a thoughtful response, but isn’t that at all. LLMs do not actually think and do not have any concept of truth.

    This is probably why things like ClickUp naming their AI tool “Brain” annoys me so much. It’s designed partially as a way for organizations to get aggregated access to the major LLMs. So yeah, my former coworkers are getting LLM output from “Clickup Brain.” What a marketing scam.

    I’ve been wondering how people’s attitudes toward LLMs would shift if society collectively changed the language we used about them to be more accurate. Maybe there wouldn’t be so many people claiming “AI is great for research” and whatnot. Even then, though, I doubt people would fully get past the human tendency to trust confident-sounding language.


  • Yes, Google’s AI features are horrifying for so many reasons!

    I can’t fathom all of the hallucinated information spread by Google alone, often to people who weren’t even trying to use AI but got an AI overview at the top of their results anyway. Google AI mode is just creating more BS that most people will never notice because they won’t check the original source of the information.



  • I agree. I think to get around 4-7, we’d need a completely different type of AI. We’d need something that isn’t an LLM, but that can do some/all of the legitimate things people are trying to use LLMs for.

    The vector search thing is nice. I used to sometimes like the automated music recommendations I got on certain streaming services, which I’m guessing worked something like that.