New era of Silicon Valley runs on networking—and not the kind you find on LinkedIn.
The tech industry funnels billions into AI data centersBoth large and small chip manufacturers are increasing innovation with regards to the technology which connects servers to each other and chips.
The networking technology is as old as the computers themselves, connecting them to allow data sharing. In the world of semiconductors, networking plays a part at almost every level of the stack—from the interconnect between transistors on the chip itself, to the external connections made between boxes or racks of chips.
Networking is a core competency of chip giants such as Nvidia Broadcom and Marvell. Some companies, however, are looking for new networking techniques to help them accelerate the huge amounts of digital data flowing through their data centers. Deep-tech companies like Lightmatter and Celestial AI use optical technology for high-speed computation.
The optical technology or photonics is experiencing a renaissance. This technology used to be considered outdated. “lame, expensive, and marginally useful,” Pete Shadbolt is the cofounder and chief science officer of PsiQuantum. He says that interest was sparked by AI 25 years ago. Shadbolt was on a panel that WIRED and Shadbolt co-hosted last week.
Some institutional and venture capitalist investors are investing billions in startups that find new methods to increase data transmission. They hope to capture the next wave of innovation, or to at least acquire a target. The investors believe the traditional interconnect technologies, based on electrons and relying on them, cannot keep up with high bandwidth AI workloads.
“If you look back historically, networking was really boring to cover, because it was switching packets of bits,” Ben Bajarin says he is the CEO of Creative Strategies, an independent research company. He has been a technology analyst for many years. “Now, because of AI, it’s having to move fairly robust workloads, and that’s why you’re seeing innovation around speed.”
Big Chip Energy
Bajarin and other experts credit Nvidia with being a pioneer in networking technology when it acquired two important companies years ago. Nvidia acquired Mellanox Technologies in 2020 for $7.6 billion. Mellanox is an Israeli company that makes networking products and solutions. Nvidia bought Cumulus Networks shortly after to power their Linux-based software system For computer networking. Nvidia was at a pivotal point, as it correctly bet that GPUs would gain in power when they were clustered together and placed into data centres.
While Nvidia dominates in vertically-integrated GPU stacks, Broadcom has become a key player in custom chip accelerators and high-speed networking technology. Its $1.7 billion company collaborates closely with Google Meta and OpenAI to develop chips for data centres. Also, it’s at the cutting edge of silicon photonics. Reuters also reported last month that Broadcom is readying a new networking chip Thor Ultra provides an ultra-high performance, lightweight and compact device. “critical link between an AI system and the rest of the data center.”
During its last earnings call, ARM revealed plans to purchase the network company DreamBig from Samsung for $265 Million. DreamBig makes AI chiplets—small, modular circuits designed to be packaged together in larger chip systems—in partnership with Samsung. The startup was founded in 2009. “interesting intellectual property … which [is] very key for scale-up and scale-out networking” Rene Haas said ARM’s Chief Executive Officer on its earnings call. It is possible to connect components by sending data down and up a chip cluster as well as connecting other chips.
Light On
Lightmatter CEO Nick Harris pointed out that the amount of computing power that AI requires now doubles every three months—much faster than Moore’s Law dictates. Computer chips continue to grow. “Whenever you’re at the state of the art of the biggest chips you can build, all performance after that comes from linking the chips together,” Harris says.
Its company uses cutting-edge technology and is not dependent on conventional networking. Lightmatter creates silicon photonics to link together chips. The company claims that it can make chips. world’s fastest photonic engine for AI chipsThe 3D silicon stack is connected using light-based technology. Over the last two years, investors such as GV and T. Rowe Price have invested more than 500 million dollars in this startup. Its valuation last year was $4.4 billion.
