How Apple fits a LLM on iPhone with surgical efficiency

1 comments

• paxy 12 hours ago ago

  TL;DR

  - Quantization

  - KV caching

  - LoRA

  - Sparse MoE model

  - Dedicated AI chip

  - Offload to cloud

  Nothing really groundbreaking.