The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM–higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.
And that’s going to he the new default platform for smart ovens or something?
Because Anthropic up there Is claiming that LLMs can propagate through IoT devices.
The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM–higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.