Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
AI-trends.todayAI-trends.today
Home»Tech»Sakana AI’s Error-Diffusion trains Dual-Stream networks that are Dale compliant, reaching up to 96.7% in MNIST or 61.7% for CIFAR-10 with no backpropagation

Sakana AI’s Error-Diffusion trains Dual-Stream networks that are Dale compliant, reaching up to 96.7% in MNIST or 61.7% for CIFAR-10 with no backpropagation

Tech By Gavin Wallace18/07/20269 Mins Read
Facebook Twitter LinkedIn Email
Samsung Researchers Introduced ANSE (Active Noise Selection for Generation): A
Samsung Researchers Introduced ANSE (Active Noise Selection for Generation): A
Share
Facebook Twitter LinkedIn Email

The brain is not able to use this mechanism, which dominates backpropagation. Specifically, backward passes require exact translations of weight matrices. It is important to note that this is not the Problems with weight transportation. Sakana AI is a new product. paper, Diffusing BlameThis constraint is faced directly by the research team. The team of researchers trains networks to obey. Dale’s principle While avoiding all weight transport.

What is error diffusion?

Error Diffusion is a rule of local learning, which was first introduced by Kaneko (2000). Three signals are all that is needed to update the weight. The three signals are the presynaptic signal, a derivative of postsynaptic stimulation, and one global error. Consequently ED doesn’t use random feedback matrices, or forward-transposed weights. Due to this locality, ED is compatible with Dale’s principal. Prior research has only shown ED to be applicable for binary classification and the MNIST.

Dual-Stream Architecture

Each layer was split into two streams by the team of researchers to meet this requirement. Two streams are excitatory.The p), aThen,d the other is inhibitory (n). The forward pass computes excitatory-minus-inhibitory preactivations for each stream:

p_i = φ_i( +p_{i-1} Wpp  − n_{i-1} Wnp  + bp )
n_i = φ_i( +n_{i-1} Wnn  − p_{i-1} Wpn  + bn )

The weight matrixes are all non-negative. Biases BThe p The following are some examples of how to get started: BThen, They do not have to be negative. The negation sign before You can also find out more about WNp The following are some examples of how to get started: You can also find out more about WPn These are not learned, but structural. Cross-stream connections are therefore inhibitory, while learnable weights continue to remain positive. This design needs four weight sub-matrices per layer. As a result, it uses roughly 4× more parameters than a single-stream network. For the same architecture, that is ∼32M versus ∼8M for DFA.

Modulo Error Routing

This architecture is the foundation for all future extensions. modulo error routing. Error Diffusion is now a more complex classification than binary. For hIdden unit i. The researchers define the routing r(i) = i mod You can also find out more about C. Here, C This is called the output dimension. This unit learns then from the component of routed errors. Essentially, every hidden unit gets a specific output channel. Unlike DFA whose random feedback matrices, ED utilizes this structured correspondence.

Three Innovations in Classification

To build on this route, The research team has added three new fixes to multi-class classification.

  1. Specific sigmoid sizes for each layer Use φi(z) = 1/(1 + e−2z/αi). Since the sigmoid derivative directly gates the error signal, attenuation is severe. In fact, post-hoc analysis reveals a 25× decay from the output to the first hidden layer. Wider sigmoids keep derivatives larger, preventing premature saturation. The team sets α = 3.0 for CIFAR-10 convolutional layers and α = 6.0 for fully connected layers.
  2. Batch-centered class error subtracts the per-class mini-batch mean. This makes the one-vs-all error zero-mean across the batch for every class. It thereby reduces persistent suppression caused by the 9:1 target imbalance.
  3. Asymmetric initialization scales excitatory weights by 1.5× and inhibitory weights by 0.5×. That gives an expected E/I scale ratio of 3:1, while the output layer stays symmetric.

Performance

With all three innovations, Error Diffusion (ED) reaches 96.7% on MNIST and 61.7% on CIFAR-10. In contrast, seed ED without them collapses to 50.4% and 11.6%. DFA scores higher on both tasks but violates Dale’s principle, using ∼2.84M negative weights. Notably, this is the first time ED has trained convolutional networks. Previously, Fujita (2026) reached ∼55.2% on CIFAR-10 using a flattened MLP. Even so, 61.7% remains far from standard gradient-based methods.

Method MNIST CIFAR-10 Dale-compliant Notes
Proposed ED 96.7% 61.7% Yes All weights non-negative; first ED on CNNs
Seed ED 50.4% 11.6% Yes No innovations; α = 1.0, raw error, symmetric init
DFA 97.6% 69.1% No Random feedback; ∼2.84M negative weights

The Ablation Reversal

Interestingly, the innovations’ importance flips between tasks. On MNIST, removing layer-specific widths is catastroPhIc (−71.4 pp), collapsIIf (mode === “ed”)If (done === hidden.length)ng accuracy toward chance. Batch-centering barely matters there (−0.3 pp). On CIFAR-10, however, the order reverses. Removing batch-centered error becomes the largest drop (−47.9 pp), collapsing four of five seeds. You can also contact us by clicking here.If you want to know more about, click here.his reversal exposes task-dependent credit-assignment bottlenecks invisible to single-benchmark evaluation.

Error Diffusion in Reinforcement Learning

Beyond classification, the research team integrate ED with Proximal Policy Optimization (PPO). They call the result ED-PPO and test it on Brax locomotion and Craftax. Here, policy-output error is routed to hidden units by output channel. For the scalar value network, the error is broadcast to all units. Importantly, ED-PPO drops the three classification innovations entirely. Across five seeds, ED-PPO beats BP-PPO on HalfCheetah (5494 vs 3520; p

Use Cases and Examples

Three settings make this concrete:

  • Neuromorphic and photonic hardware often encodes non-negative synaptic magnitudes physically. ED’s fixed-sign routing maps cleanly onto such substrates, complementing prior photonic DFA work.
  • The non-negative floor drives 37.3% of weights to the floor (10⁻⁴) after training. Inhibitory cross-stream fully connected connections are pruned most, up to 68.8%. This implicit sparsity hints at model compression “for free.”
  • The dedicated inhibitory stream may help continual and open-ended learning. It provides a structural mechanism for dampening large gradient excursions.

Comparison

How Dale-Compliant Error Diffusion Compares

Proposed approach vs. other backpropagation-free and biologically motivated learning rules. “Dale-compliant” means separate excitatory/inhibitory populations with non-negative weights. Method names link to primary sources.

Takeaway: most backprop-free rules relax Dale’s principle, and most Dale-compliant networks still rely on backpropagation. Error Diffusion satisfies both at once, and is the first to carry that combination into reinforcement learning.

Minimal Code Sketch

The equations translate into a compact, illustrative update loop:

import torch

def dual_stream_forward(p, n, Wpp, Wnp, Wnn, Wpn, bp, bn, phi):
    # All W >= 0; cross-stream signs are hardcoded inhibitory (Dale's principle)
    p_next = phi(p @ Wpp - n @ Wnp + bp)   # excitatory stream
    n_next = phi(n @ Wnn - p @ Wpn + bn)   # inhibitory stream
    return p_next, n_next

def routed_error(S, H, C):                 # S: output error, shape (B, C)
    M = torch.zeros(H, C)
    for i in range(H):
        M[i, i % C] = 1.0                  # r(i) = i mod C
    return S @ M.T                         # R = S M^T,  shape (B, H)

def ed_update(A_p, Z_p, R, phi_deriv):
    U_p = phi_deriv(Z_p) * R               # local postsynaptic drive
    return A_p.T @ U_p                     # dWpp  ∝  A_p^T U_p, shape (K, H)

Key Takeaways

  • Sakana AI’s Error Diffusion trains Dale-compliant dual-stream networks without weight transport or random feedback matrices.
  • Modulo error routing (r(i) = i mod C) scales the rule past binary classification to 96.7% MNIST and 61.7% CIFAR-10.
  • The importance of three classification innovations has reversed between MNIST & CIFAR-10 highlighting task-dependent credit assignment bottlenecks.
  • ED-PPO uses the same reinforcement learning architecture as DFA-PPO, but is superior on Craftax.
  • Dale’s principle costs 0.9–7.4 points versus DFA on classification, quantifying the price of non-negative weights.

Click here to find out more Paper. Also, feel free to follow us on Twitter Don’t forget about our 150k+ML SubReddit Subscribe now our Newsletter. Wait! What? now you can join us on telegram as well.

You can partner with us to promote your GitHub Repository OR Hugging Page OR New Product Launch OR Webinar, etc.? Connect with us


Asif Razzaq serves as the CEO at Marktechpost Media Inc. As an entrepreneur, Asif has a passion for harnessing Artificial Intelligence to benefit society. Marktechpost was his most recent venture. This platform, which focuses on machine learning and deep-learning news, is unique for the way it is presented in a technical manner, but is also easily understood by all audiences. Over 2 million views per month are a testament to the platform’s popularity.

AI ar Net work
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

Take the Quiz! Will AI destroy your career?

Senator Blackburn Pulls Support for AI Moratorium in Trump’s ‘Big Beautiful Bill’ Amid Backlash

‘Tell Him He’s a Piece of Shit’: Meta’s New AI Unit Is a Total Mess

Age Verification Is Sweeping Gaming. Are you ready for AI Fakes in the Age of Gaming?

Infiltrated Moltbook: the AI-Only Social Network, where humans are not allowed

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

The Body Betraying Privacy

24/03/2026

Google DeepMind introduces Aeneas, AI-powered contextualization and restoration of ancient Latin Inscriptions

26/07/2025
Latest News

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026

OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.