2023年12月4日 · Piping hot off the virtual presses is llamafile, a new framework that let’s you run Large Language Models (LLMs) on your local machine on a single file. The aim of the project is to make LLMs more accessible and easy to distribute, but to me, the clearest most immediate value is being able to run an LLM off of a USB.

2025年2月17日 · LLMs designed for limited hardware or consumer-grade PCs are available now as well, but [Binh] wanted something even smaller and more portable, so he put an LLM on a USB stick. This USB stick...

2025年2月14日 · A Raspberry Pi Zero W has been modified to function as a USB stick running a local large language model (LLM) entirely offline. This innovative device, created by Binh Pham from Build With Binh, interacts with users through USB mass storage, allowing AI-generated content to be written dynamically by simply creating a file.

2025年3月14日 · Model Context Protocol (MCP) 是一种开放协议，它实现了 LLM 应用程序与外部数据源和工具的无缝集成。 该协议由 Anthropic 于 2024 年 11 月 25 日发布。 与 USB 的类比 MCP 可以被视为 AI 系统的「USB 标准」。

A llamafile is an executable LLM that you can run on your own computer. It contains the weights for a given open LLM, as well as everything needed to actually run that model on your computer.

2025年2月20日 · Youtuber and tech enthusiast Binh Pham has recently built a portable plug-and-play AI and LLM device housed in a USB stick called the LLMStick and built around a …

💪 I created the first USB in the world that has LLM running locally on it.Cherry on top of the cake, it requires no dependency, you can connect it to any co...

2023年4月16日 · LLMs are super memory bound, so you'd have to transfer huge amounts of data in via USB 3.0 at best. Just for example, Llama 7B 4bit quantized is around 4GB. USB 3.0 has a theoretical maximum speed of about 600MB/sec, so just running the model data through it would take about 6.5sec.

2025年2月17日 · By plugging the stick into a computer, one can interact with the LLM by simply creating a text file — no technical skills are required. Inside the 3D-printed shell of this somewhat-oversized USB stick is a Raspberry Pi Zero single-board computer and a shield that adds a male USB port for interfacing with a host computer.

2024年7月28日 · Connect home devices into a powerful cluster to accelerate LLM inference. More devices mean faster performance, leveraging tensor parallelism and high-speed …