mHC: Manifold-Constrained Hyper-Connections
TL;DR
Imagine building with LEGOs. A simple, deep tower (a basic neural network) can get wobbly and fall. Someone invented a special LEGO piece (a 'residual connection') that acts like a super-strong internal support beam, letting you build much taller, stable towers. Then, another builder tried adding lots of extra crisscrossing beams ('Hyper-Connections') for even more strength, but this made the whole structure complicated and surprisingly unstable again. This paper introduces a new, smarter way to add those extra beams ('mHC'). It's like using precisely engineered brackets that add strength without messing up the main support structure, resulting in the tallest, strongest, and most stable tower yet.
Recently, studies exemplified by Hyper-Connections (HC) have extended the ubiquitous residual connection paradigm established over the past decade by expanding the residual stream width and diversifying connectivity patterns. While yielding substantial performance gains, this diversification fundamentally compromises the identity mapping property intrinsic to the residual connection, which causes severe training instability and restricted scalability, and additionally incurs notable memory access overhead. To address these challenges, we propose Manifold-Constrained Hyper-Connections (mHC), a general framework that projects the residual connection space of HC onto a specific manifold to restore the identity mapping property, while incorporating rigorous infrastructure optimization to ensure efficiency. Empirical experiments demonstrate that mHC is effective for training at scale, offering tangible performance improvements and superior scalability. We anticipate that mHC, as a flexible and practical extension of HC, will contribute to a deeper understanding of topological architecture design and suggest promising directions for the evolution of foundational models.
- 1Proposes a framework to project residual connections onto a manifold to restore identity mapping.
- 2Addresses training instability and scalability issues of Hyper-Connections.
- 3Demonstrates superior scalability and performance improvements of mHC.
Over 20,000 precolonial earthworks in the Southwest Amazonia
Imagine flying a special laser scanner over the Amazon jungle that can 'see through' the treetops, like X-ray vision for the ground. When scientists did this, they found over 20,000 geometric shapes — ditches, mounds, and enclosures — built by ancient people long before Europeans arrived. These aren't small things: they're massive earthen structures, like monuments. This means the Amazon rainforest, which most people picture as empty wilderness, was actually home to millions of people who built cities and shaped the landscape. Think of it like discovering that a forest you thought was wild was actually someone's ancient garden on a continental scale.
A digitally controlled silicon quantum processing unit
Imagine you want to build a super-powerful calculator that uses the weird rules of quantum physics to solve problems no regular computer can. The trouble is, the tiny quantum pieces — called qubits — are incredibly fragile and need to be kept colder than outer space. On top of that, you need wires and control signals going to every single qubit, and if you have thousands of them, the wiring becomes a nightmare. This team solved part of that puzzle by building their qubits out of silicon (the same stuff in your phone's chip), adding a tiny control computer that works at super-cold temperatures right next to the qubits, and using a special high-density cable to connect everything cleanly. They packed 54 tiny quantum dots onto a chip, arranged 18 of them into working qubits, and showed the qubits work about 10 times better than any previous silicon qubit of this type. They also ran basic error-correction experiments to prove the system is on track for real-world use.
The 2026 World Cup's grass is an engineering problem
Imagine you're trying to play soccer in 16 different places across the United States, Canada, and Mexico — some in freezing cold, some blazing hot, some in stadiums with roofs that block sunlight. Half of those stadiums normally use fake grass. Now FIFA, the organization that runs the World Cup, wants every single pitch to feel and play exactly the same way, like a video game where every level has identical physics. To do that, they hired grass scientists — yes, that's a real job — who figured out how to grow special grass on thin mats with plastic underneath so it can be transported like a carpet, stitched with synthetic fibers so it doesn't rip when players sprint and tackle, and tested by literally shooting balls at it with a cannon to make sure it bounces right. Different grass species are used depending on whether a stadium is hot, cool, or dark. It's basically a giant, living, high-tech floor installation that has to survive the world's best athletes running on it.
Remarks on the disproof of the unit distance conjecture
Imagine you scatter a bunch of dots on a piece of paper. The question is: how many pairs of those dots can be exactly 1 inch apart? The Erdős unit distance conjecture asked whether there's a specific mathematical formula that limits how often this can happen as you add more and more dots. Think of it like asking how many friendships can exist in a town where friends are defined as people who live exactly one mile apart — there's a suspected maximum, and Erdős guessed what that maximum should be. For decades, no one could prove or disprove his guess. Now, an AI apparently found a specific arrangement of dots (a 'counterexample') that breaks the expected limit, proving Erdős's conjecture was wrong. A team of elite mathematicians then checked and explained the AI's work in this paper.