I think neuro-symbolic AI has a lot of potential here, since small models can handle a lot of conversational inputs, while relying on wired-in solvers for more complex symbolic math/computation needs. https://jjd.io/posts/swollm-bbh-leaderboard.html
I imagine that for most "emergency" scenarios, a device that stores large amounts of prepared normal reference material (encyclopedia, guides, howtos, etc.) will be dramatically cheaper, more durable, and able to run usefully on limited power supplies.
If someone will need something like a medical flowchart for first-aid, it's probably better to generate that kind of artifact in advance and then get it verified by an expert.
Oh, it might make sense someday, but for the forseeable future I don't think it'd be sane to buy a multi-GPU computer, ruggedize it, then tuck it away to depreciate in a "safe" spot just in case the big earthquake/flood/EMP happens... Only to find out that there's nothing that can supply its power needs and that it's too heavy to relocate.
This is couched in prepper nonsense, but it's got LLM, WikiPedia, maps, etc. A bunch of genuinely useful stuff to keep on a USB stick or whatever: https://www.projectnomad.us/
But, the current model you really want for an emergency kit is Gemma 4 12B QAT 4-bit. At ~7GB on disk, it's small enough to run on a tablet or any modern computer, slowly if you don't have a GPU or modern Apple silicon, but exceedingly smart for its size, excellent vision capabilities, good tool user, surprisingly good reasoning.
I've been mulling over a good use of a large philanthropy spend in the next decade, and I would love to build a bunch of hardware "oracles" that include an LLM. Ideally solid state, visual/audio, solar + usb-c, so, good in a lot of doomsday scenarios as well as just out hiking. It's a fun thought experiment. I imagine making like 1 million of them, they could be sold and genuinely useful, but also stored and put in an emergency box.
They want to ask the iOS Foundation model (frontier on device intelligence for something small) for instance about emergency procedures and life-saving info. I wouldn’t trust that model with much at all though. More likely to find what you need from miniature survival guides.
99% of the model "work" (meaning the connection to your computer) is just spinning a spinner - something that makes me want to wrap it with a mosh shell so I can just keep moving from network to network.
I think neuro-symbolic AI has a lot of potential here, since small models can handle a lot of conversational inputs, while relying on wired-in solvers for more complex symbolic math/computation needs. https://jjd.io/posts/swollm-bbh-leaderboard.html
Is anyone making LLM-in-a-box for emergency supply kits yet?
I feel that would be handy in all sorts of situations when networks are down.
> emergency supply kits
I imagine that for most "emergency" scenarios, a device that stores large amounts of prepared normal reference material (encyclopedia, guides, howtos, etc.) will be dramatically cheaper, more durable, and able to run usefully on limited power supplies.
If someone will need something like a medical flowchart for first-aid, it's probably better to generate that kind of artifact in advance and then get it verified by an expert.
Oh, it might make sense someday, but for the forseeable future I don't think it'd be sane to buy a multi-GPU computer, ruggedize it, then tuck it away to depreciate in a "safe" spot just in case the big earthquake/flood/EMP happens... Only to find out that there's nothing that can supply its power needs and that it's too heavy to relocate.
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?
This is couched in prepper nonsense, but it's got LLM, WikiPedia, maps, etc. A bunch of genuinely useful stuff to keep on a USB stick or whatever: https://www.projectnomad.us/
But, the current model you really want for an emergency kit is Gemma 4 12B QAT 4-bit. At ~7GB on disk, it's small enough to run on a tablet or any modern computer, slowly if you don't have a GPU or modern Apple silicon, but exceedingly smart for its size, excellent vision capabilities, good tool user, surprisingly good reasoning.
I've been mulling over a good use of a large philanthropy spend in the next decade, and I would love to build a bunch of hardware "oracles" that include an LLM. Ideally solid state, visual/audio, solar + usb-c, so, good in a lot of doomsday scenarios as well as just out hiking. It's a fun thought experiment. I imagine making like 1 million of them, they could be sold and genuinely useful, but also stored and put in an emergency box.
Can you expand what you mean?
They want to ask the iOS Foundation model (frontier on device intelligence for something small) for instance about emergency procedures and life-saving info. I wouldn’t trust that model with much at all though. More likely to find what you need from miniature survival guides.
99% of the model "work" (meaning the connection to your computer) is just spinning a spinner - something that makes me want to wrap it with a mosh shell so I can just keep moving from network to network.
Fascinating to wonder whether the bigger model finds fewer or more counterfeits than the on-device one.