Apple is developing a way to run LLMs on iPhones

Apple’s AI developers say they’ve made significant progress in deploying large language models (LLMs, like ChatGPT) on iPhones and other Apple devices with limited memory, thanks to an innovative new technique for using flash memory.

LLM programs such as ChatGPT and Cloud typically require large amounts of memory to run, which presents a challenge for iPhones with limited memory capacity. To address this problem, Apple developers have come up with a new technology that uses flash memory — the same memory where your apps and photos live — to store AI model data, Macrumors writes.

In a new paper titled LLM in a Flash: Efficient Large Language Model Inference with Limited Memory, the authors write that flash storage is larger on mobile phones than working memory, which is traditionally used to run LLM. Their new method overcomes this limitation with two key technologies that minimize data transfer and maximize flash storage throughput:

Windows: Think of this as a way to recycle. Instead of loading new data each time, AI reuses some of the data it has already processed. This reduces the need to constantly retrieve memory, making the process faster and smoother.

Linking rows and columns: This technique is similar to reading a book in larger chunks rather than one word at a time. By collecting data more efficiently, it can be read faster, accelerating AI’s ability to understand and generate language.

Combining these approaches allows AI models to use twice as much working memory as the phone actually uses.

This breakthrough opens up new possibilities for future iPhones, such as more advanced Siri functionality, real-time language translation, and advanced AI-powered features in photography and augmented reality.

Apple is reportedly in the process of developing its own artificial intelligence called “Ajax.”