Because the companies that provide these models/apps aren’t doing a service to the world. Their only motive is to make as much money as possible. I think that this is their idea to achieve that:

1. Create an AI model that’s excellent and free.
2. Users use it and since they like it, they continue using it.
3. Use the chats to train the next generation of models and make it more powerful.
4. Ensure that a lot of people use AI so frequently they get addicted to it because they feel its so good and useful.
5. Once enough market share is captured, place everything behind a paywall.
6. Everyone who is addicted to AI will pay because they can’t live without it.
7. Profit

At least, this is probably Sam Altman’s dream.

I saw this one person who, despite getting 3 warnings (one in the chat, 2 in the file itself) about not placing a plaintext API key into a version-controlled env file from a chatbot, did so anyway. Its not just about the AI, but also about the people using them. Someone with experience will be able to utilize the speed of an AI, while finding its mistakes as well. A “vibe coder” won’t know the difference.