Theme
AI Resources
Edge0
Edge0 is a local inference framework that streams mixture-of-experts model weights from SSD instead of keeping the whole checkpoint active in memory.
Its two published tiers pair 4-bit model checkpoints with trained LoRA and routing adapters. The current MLX backend runs on Apple Silicon and can serve either model through a local OpenAI-compatible chat endpoint. Use this as a first read, not a recommendation. Open the original project before trusting details like terms, limits, privacy, cost, setup, or safety.
What it is
A streaming runtime for sparse models
Edge0 keeps expert weights on storage, predicts which experts the next token will need, and loads that smaller active set while generation continues.
Why it stands out
Memory and storage are treated separately
The 35B-class checkpoint is about 23 GB on disk, but the project reports roughly 2.9 GB of peak active memory at short contexts on its M4 Pro test system. The smaller 8B-class tier is about 4.2 GB on disk and was measured at roughly 1 GB of active memory.
Availability
Public framework, checkpoints, and adapters
The Apache-2.0 repository includes installation instructions, tests, model documentation, a Python API, and a local server. Both preview checkpoints are available from the project's Hugging Face account with their trained adapters.
Why it matters
What makes it useful
Sparse MoE models use only a subset of their experts for each token. Edge0 builds its runtime around that gap between the complete checkpoint on storage and the smaller active set in memory, giving Apple Silicon builders another way to test a larger expert model.
What to know
Where it fits
It fits technical local-AI experiments on a Mac with fast internal storage, especially when the full checkpoint is larger than the memory footprint a builder wants to keep active. It is a framework plus two matched preview checkpoints, not a general runtime for arbitrary models or a finished consumer chat app.
Notable points
What stands out
Edge0 publishes its own quality and performance tables. The reported memory and speed figures come from a Mac mini M4 Pro with specific prompts and settings, so treat them as project measurements rather than results for every Apple Silicon machine or workload.
Before using
What to review
The current backend supports macOS on Apple Silicon only. CUDA is listed as future work, and Windows, Linux, and non-Apple hardware are not supported by the released backend.
Use the project's pinned MLX versions. The current requirements list mlx 0.30.6, mlx-metal 0.30.6, and mlx-lm 0.31.0, and the repository warns that mlx 0.31.2 or newer currently breaks its streaming loader.
Plan for the complete model download even though active memory is lower. The published checkpoints are about 4.2 GB for the 8B-class tier and 23 GB for the 35B-class tier.
Longer contexts grow the KV cache. The project's lowest memory figures are for short contexts, not a fixed ceiling for every conversation.
The checkpoints are early previews. The model cards say agentic tool use, multi-step planning, and long-horizon autonomy are not yet strengths of this release.
Use the checkpoint, LoRA adapter, and prerouter adapter as a matched set for the published Edge0 pipeline. The documentation also allows either adapter to be disabled when running the plain base model.
Reader fit
Who may find it relevant
Apple Silicon users exploring local sparse-model inference with limited active memory.
Builders comparing SSD expert streaming with CPU-GPU offload or fully memory-resident serving.
Developers who want a Python API or local OpenAI-compatible chat endpoint around the released checkpoints.
Less suitable for people who need broad hardware support, arbitrary model compatibility, or a polished desktop chat app.
Editorial note
Why LifeHubber lists it
Edge0 makes a normally hidden tradeoff easy to inspect: a model can occupy much more storage than the expert weights it keeps active for each token. Its two matched checkpoints give Mac builders a concrete way to compare SSD streaming with other low-memory inference approaches, while the preview status and Apple-only backend keep the choice grounded.
Source links
Source materials
Reader note
Before relying on this entry
LifeHubber lists entries to help readers inspect AI projects, not to endorse them or prove they are safe, suitable, accurate, maintained, or right for a specific use. We do not verify every entry in depth. Before relying on anything listed, review the original materials, terms, privacy practices, limits, and risks that matter for your situation.
What to explore next
Compare another way to run large MoE models on one machine.
Edge0 streams predicted expert weights from SSD on Apple Silicon. FreeToken takes a broader NVIDIA-and-CPU route, making the hardware and offload tradeoff easier to compare.
More in Ecosystem
Keep browsing this category
Explore more AI ecosystem resources.
LEANN
StarTrail-org/LEANN
A local vector index for semantic search and personal RAG that reduces stored embeddings through selective recomputation, with Python, CLI, and MCP routes.
MiniMind
jingyaogong/minimind
A native-PyTorch learning project for training tiny language models from scratch, with public data and checkpoints, tokenizer, pretraining and alignment stages, LoRA, tool-use and agentic-RL examples, plus local inference paths.
HyperFrames
heygen-com/hyperframes
A video rendering framework for HTML-based compositions, positioned around agent-friendly workflows, previewing, and MP4 rendering.
For project maintainers
Listed here? You can use the badge.
If you maintain a project with a current LifeHubber listing, you may add the optional “Listed on LifeHubber AI Resources” badge to its README, docs, or website. No introduction or permission request is needed.
