IBM researchers, together with ETH Zürich, have unveiled a new class of Analog Foundation Models (AFMs) The aim of this project is to provide a bridge between the large language models and Analog In Memory Computing hardware. AIMC has long promised a radical leap in efficiency—running models with a billion parameters in a footprint small enough for embedded or edge devices—thanks to dense non-volatile memory (NVM) that combines storage and computation. The Achilles heel of the technology has always been noise. Matrix-vector multiplications performed directly within NVM devices result in non-deterministic error that can cripple commercial models.
Why is analog computing important for LLMs
AIMC is a matrix-vector multiplier that performs the multiplication directly within memory arrays. It eliminates the von Neumann bottleneck while delivering massive increases in both throughput and energy efficiency. Prior studies revealed that AIMC combined with 3D NVM The following are some examples of how to get started: Mixture-of-Experts (MoE) The architectures would be able to support in theory models of trillion parameters, even on small accelerators. This could allow AI to be implemented on devices outside of data centers.
What is the Difference Between Analog in-Memory computing (AIMC is it so difficult to use practically?
Noise is the biggest obstacle. AIMC calculations are affected by device variability, DAC/ADC quantumization and runtime fluctuation that reduce model accuracy. Unlike quantization on GPUs—where errors are deterministic and manageable—analog noise is stochastic and unpredictable. Researchers have found ways of adapting small networks such as CNNs and RNNs.
Analog Foundation Models: How can they address noise?
IBM introduces its team Analog Foundation ModelsThis training integrates hardware-awareness to prepare LLMs on analog execution. They use:
- Noise injection During training, AIMC randomness is simulated.
- The weight is cut in a series of iterative steps Stabilize distributions to device limits
- Learned ranges of static input/output Quantization Aligned to real hardware constraints
- Distillation using pre-trained LLMs Synthetic data 20B tokens can be used.
The following methods are implemented in conjunction with AIHWKIT-LightningModels like Phi-3-mini-4k-instruct The following are some examples of how to get started: Llama-3.2-1B-Instruct The ability to endure Under analog noise, performance is comparable to baselines of weighted 4-bit and activation 8 bit.. AFMs performed better than both Quantization-aware Training (QAT), and Post-Training Quantization (SpinQuant) in evaluations of reasoning and factual standards.
These models only work with analog hardware
No. AFMs perform well on low-precision digital hardware. AFMs can handle round-to nearest quantization (RTN) better than other methods because they are trained to deal with noise. They are therefore useful for both AIMC acceleration hardware and commodity digital inference equipment.
How much compute can you add to the inference process?
Yes. Researchers tested test-time compute scaling On the MATH500 benchmark, the model generates multiple responses per question and selects the best using a reward-based system. AFMs exhibited better scaling behaviour than QATs, and accuracy gaps decreased as more compute for inference was allocated. This is consistent with AIMC’s strengths—low-power, high-throughput inference rather than training.
What is the future of Analog In-Memory Computing?
The team of researchers provides the first systematized demonstration that LLMs large than a few kilobytes can be adapted for AIMC hardware with no catastrophic loss in accuracy. Although training AFMs can be resource-intensive, and tasks like GSM8K have accuracy gaps even after training is complete, these results represent a significant milestone. This combination is a milestone. Cross-compatibility, energy efficiency and robustness against noise AFMs offer a way of scaling models above the GPU limit.
The following is a summary of the information that you will find on this page.
Analog Foundation Models are a crucial step in scaling LLMs past the limitations of digital accelerators. AIMC has become a real platform by making models resilient to unpredictable noises of analog in memory computing. The training costs and reasoning benchmarks are still high. However, this research establishes the path towards energy-efficient, large scale, models running on compact, low cost hardware. It also pushes foundation models to closer edge deployment.
Take a look at the PAPER The following are some examples of how to get started: GITHUB PAGE. Check out our website to learn more. GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter Join our Facebook group! 100k+ ML SubReddit Subscribe now our Newsletter.
Asif Razzaq, CEO of Marktechpost Media Inc. is a visionary engineer and entrepreneur who is dedicated to harnessing Artificial Intelligence’s potential for the social good. Marktechpost was his most recent venture. This platform, which combines a technical and understandable coverage of deep learning and machine learning news, is renowned for its comprehensive and in-depth reporting. This platform has over 2,000,000 monthly views which shows its popularity.
