Memory Prices Indefinitely High: SK Hynix Says Shortage Continues Through 2030 We haven't been complaining about RAM prices quite as much on the podcast
Self hosting isnt a threat to them, most folks ain’t doing that. And those of us who do already block most of their ways to generate revenue of our usage, so we don’t really matter to them.
Self hosting also requires a ton of RAM because each individual user has to load a (usually slightly smaller) model + KV cache, whereas for hosted, they load the huge-ass model once (per cluster) and then store KV caches for a bunch of users, which has better scaling.
The memory companies win whether you self-host or pay a subscription. They might win more if you self-host because look at the cost of self-hosting, it’s actually pretty steep if you don’t have the hardware already from the before times.
I don’t like conspiracy theories but hugging face was bought, tens of thousands of Mac’s were bought, RAM is “indefinitely “ high priced.
Starting to feel someone doesn’t want the self hosted AI to get legs.
It’s just hyper capitalism unfortunately.
Self hosting isnt a threat to them, most folks ain’t doing that. And those of us who do already block most of their ways to generate revenue of our usage, so we don’t really matter to them.
Self hosting also requires a ton of RAM because each individual user has to load a (usually slightly smaller) model + KV cache, whereas for hosted, they load the huge-ass model once (per cluster) and then store KV caches for a bunch of users, which has better scaling.
The memory companies win whether you self-host or pay a subscription. They might win more if you self-host because look at the cost of self-hosting, it’s actually pretty steep if you don’t have the hardware already from the before times.