Scientists have built a “thermodynamic computer” that can generate images from random disturbances, or noise, in data. In doing so, they mimicked the generative artificial intelligence (AI) capabilities of neural networks, collections of machine learning algorithms modeled after the brain.
Above absolute zero, the world becomes noisy with energy fluctuations called thermal noise. These fluctuations occur when atoms and molecules wiggle, or when the direction of the quantum properties that give them magnetism reverses on an atomic scale.
Today’s AI systems, like most other computer systems, use computer chips to generate images. The energy required to flip a bit is small compared to the amount of energy contained in random fluctuations in thermal noise, and the noise is negligible.
you may like
But new “generative thermodynamic computers” work with noise in the system rather than ignoring it, meaning they can complete computing tasks using orders of magnitude less energy than typical AI systems require. Scientists outlined their findings in a new study published January 20 in Physical Review Letters.
Stephen Whiterum, a staff scientist at Lawrence Berkeley National Laboratory’s Molecular Foundry and author of the new study, compared it to a boat at sea. Waves act as thermal noise here, and traditional computing can be “likened to a cruise ship passing by obliviously. It’s very efficient, but it’s very expensive,” he said.
But trying to reduce the energy consumption of traditional computing to an energy consumption comparable to thermal noise is like trying to steer a dinghy across an ocean with an outboard engine. “It’s much harder,” he told Live Science, but harnessing noise in thermodynamic computing could be useful in the same way “surfers harness the power of waves.”
Traditional computing operates using clear binary bit values (1s and 0s). However, a growing body of research over the past decade has revealed that using value probabilities instead provides more benefits in terms of resources such as power consumed to complete the calculation.
The efficiency gains are particularly noticeable for a particular type of problem known as an “optimization” problem. This problem requires you to get the most bang for your buck with the least amount of effort. For example, visit the most streets to deliver mail while walking the fewest miles. Thermodynamic computing can be thought of as a type of stochastic computing that uses random fluctuations from thermal noise to power calculations.
Image generation using thermodynamic computing
Researchers at the Normal Computing Corporation in New York, who were not directly involved in the image-generating work, used a network of circuits linked by other circuits to build something close to a thermodynamic computer, all operating at low energy comparable to thermal noise. The circuits that perform the linking can be programmed to strengthen or weaken the connections formed between the linking circuits (the “node” circuits).
When we apply some voltage to the system, various nodes are assigned a series of voltages that eventually decrease when the applied voltage is removed and the circuit returns to equilibrium.
you may like
However, even in equilibrium, noise in the circuit causes the values of the nodes to fluctuate in a very specific way, determined by the strength of the programmed connections, the so-called coupling strength. Therefore, the bond strength can be programmed in such a way that the resulting equilibrium fluctuations effectively raise questions to be answered. Researchers at Normal Computing have shown that the strength of the connections can be programmed so that the resulting variations in the equilibrium nodes can solve linear algebra.
Managing these connections gives you some control over what questions the equilibrium fluctuations in node values answer, but does not provide a way to change the type of question. Whiterum thought that moving away from thermal equilibrium might help researchers design computers that could answer fundamentally different kinds of questions, and might also be more useful because it takes longer to reach equilibrium.
Whiterum found himself considering research from around the mid-2010s when considering what kinds of computations could be made by moving away from equilibrium. The study showed that if you take an image and add noise until no trace of the original image is visible, you can train a neural network to reverse the process and retrieve the image. Training a neural network on a range of these missing images allows the neural network to generate a variety of images from a random noise starting point, including some images outside the library it was trained on. To Whiterum, these diffusion models seemed to be a “natural starting point” for thermodynamic computers, and diffusion itself was considered a statistical process with roots in thermodynamics.
Whiterum pointed out that while traditional computing works in a way that reduces noise to negligible levels, many of the algorithms used to train neural networks work by adding noise back in. “Wouldn’t that be much more natural in a thermodynamic setting where you get noise for free?” he noted from the meeting minutes.
borrow old principles
How things develop under the influence of significant noise can be calculated from the Langevin equation, which dates back to 1908. By manipulating this equation, we can find the probability of each step in the process that an image is swathed in noise. In a sense, when an image is affected by thermal noise, you get a probability that each pixel will flip to the wrong color.
From there, you can calculate the required coupling strength (e.g., circuit connection strength) and reverse the process to remove noise step by step. This will generate the image. This was demonstrated by Whitelam in numerical simulations from a library of images containing ‘0’, ‘1’ and ‘2’. The images generated can be images from the original training database or some kind of hypothesis, and the added bonus of imperfections in the training means that new images may be generated that are not part of the original dataset.
Ramy Sherbaya, CEO of Quantum Dice, a company that makes quantum random number generators, who was not involved in the research, called the discovery “important.” He mentioned specific areas where traditional methods are starting to struggle in order to meet the ever-increasing demand for more powerful models. Since Sherbaya’s company makes a type of probabilistic computing hardware that uses quantum-generated random numbers, he found it “encouraging to see an increasing interest in probabilistic computing and the various computing paradigms that are closely related to it.”
He also warned of potential benefits beyond energy savings. “This article also shows how a physics-inspired approach can provide a clear fundamental interpretation in a field dominated by ‘black box’ models, and provide important insights into the learning process,” he told Live Science via email.
As generative AI advances, getting three learned numbers from noise may seem relatively rudimentary. However, Whiterum pointed out that the concept of thermodynamic computing is still only a few years old.
“Looking at the history of machine learning and how it was eventually scaled up to larger and more impressive tasks, I’m curious to see if thermodynamics hardware can scale up in the same way, even in a conceptual sense,” he said.
Source link
