Making sense of the nonsensical: Black holes and the simulation library

a row of super computers

A view between two rows of Frontera servers in the Texas Advanced Computing Center data center. Frontera is the fastest academic supercomputer in the world. UArizona's Chi-Kwan "C.K." Chan is the principal investigator the EHT-Frontera Large-Scale Community Partnerships, which provides more than 80 million core hours for the EHT to simulate black hole.

Texas Advanced Computing Center

After mobilizing more than 300 scientists and engineers to establish a network of synchronized telescopes that form an Earth-sized virtual telescope, the international Event Horizon Telescope Collaboration snapped the first-ever images of supermassive black holes. The first image, of the black hole at the center of the Messier 87 galaxy, was released in 2019. The latest image, released Thursday, shows the black hole at the center of our own Milky Way galaxy, called Sagittarius A*. 

But what happens after these images are captured?

Chi-Kwan Chan

Chi-Kwan Chan

Chris Richards/University of Arizona

"Snapping an image is just the beginning. To really understand the object we're observing, we had to compare it to simulations," said Chi-Kwan "CK" Chan, a University of Arizona associate research professor in the College of Science's Steward Observatory. Chan serves as the secretary of the EHT Science Council and is a senior investigator for the international Black Hole PIRE Project, which works to develop the infrastructure to usher astronomical projects like EHT into the era of big data science.

Chan is also a leader of the EHT collaboration's theoretical modeling and interpretation efforts for Sagittarius A*, the subject of the latest photograph and a round scientific papers published by the EHT Collaboration in Astrophysical Journal Letters. He coordinated the fifth paper, which focuses on creating black hole simulations and turning them into synthetic images that can be compared with real observations to teach us something new about the black hole.

As a result of this process, EHT scientists determined that Sagittarius A* is likely spinning and has a magnetic field slightly stronger than a refrigerator magnet, which is enough to push away nearby gas. The gas falling into the black hole forms a disk that, from Earth, appears to be face-on rather than from the edge. This diffuse glowing disk is made up of super-heated gas, or plasma, and charged particles. The electrons are 100 times cooler than the ions in the plasma, and the disk rotates in the same direction the black hole spins. Also, only some of this material falls into the black hole. If Sagittarius A* was a person, it would consume a single grain of rice every million years.

A black hole

This is the first image of Sagittarius A*, the supermassive black hole at the center of our galaxy.

EHT Collaboration

Finding meaning

UArizona, together with the University of Illinois and Harvard University, led the effort to create the biggest collection of simulations to date, which EHT calls the simulation library. This library is made up thousands of data sets – containing information about how the plasma interacts with magnetic fields around black holes – and millions of simulated images. Each simulation assumes something different about the properties and characteristics of the black hole and its surrounding environment.

EHT scientists can compare each simulated image with the actual black hole image to find a match. The simulation that creates the snapshot with the closest match can teach us something about the actual black hole, including its plasma temperature and the strength of its magnetic field.

The simulation process involves using supercomputers to solve what's called general relativistic magnetohydrodynamic – or GRMHD – equations, which reveal the movement of material and energy around black holes within dramatically warped space and time. GRMHD simulations are similar to simulations used to understand how air flows around aircrafts, Chan said, but GRMHD simulations also factor in extreme forces of gravity as described by Einstein's theory of general relativity and the interaction between magnetic fields and plasma.

Unlike simpler equations, which can be solved with pencil, paper and time, GRMHD equations are much more complex, as they account for the constant feedback between magnetic fields and plasma, resulting in an ever-changing equation.

To create the simulation library, the EHT Collaboration needed 80 million CPU hours, or processing time, which is the equivalent of running 2,000 laptops at full speed for a full year. The collaboration ran the calculations to create the library with the National Science Foundation-funded Frontera supercomputer at the Texas Advanced Computing Center, where Chan is principal investigator of the Frontera Large-Scale Community Partnerships allocation. With this resource, the team was able to finish the full library of simulations in two months.

"To compare simulations like this with EHT observations, we need to run additional calculations to translate the GRMHD data into images, too," Chan said. "Those kinds of calculations are called general relativistic ray tracing." 

The EHT was designed to detect a specific wavelength – 1.3 millimeters – of radio wave from the galactic center of a black hole. To simulate these radio waves and create images, scientists trace the path that light traveled back to the black hole, again using supercomputers. 

Chan led much of the ray tracing calculation efforts for Sagittarius A* through CyVerse, a national cyberinfrastructure based at UArizona, and the NSF-funded Open Science Grid, a consortium for the computation of large amounts of data. The UArizona team not only spearheaded the effort to acquire the computational resources to run these simulations, but they also created the software that facilitated the calculations.

The final product is many simulated movies and simulated images of a black hole produced by different assumptions about the underlying physics. The team then compares those movies and images with real black holes.

More to learn

UArizona students played an important role in making the comparison possible. Yuan Jea Hew, a recent graduate who studied astronomy, and Anthony Hsu, a sophomore studying computer science and applied mathematics, developed data analysis algorithms to make comparison possible.

The collaboration relied on 11 different tests that the black hole simulations had to pass in order to sufficiently match the real black hole.

"It is remarkable that we understand Sagittarius A* so well that we have some models pass 10 out of the 11 tests," Chan said.

The tests considered variables such as brightness of certain wavelengths, image size, and the size and width of the glowing ring surrounding the black hole.

"However, no single model passed all 11 tests," Chan said. The test that was hardest for the models to beat was the variability, which measures how much the black hole changes from moment to moment. The simulations are more variable than the real Sagittarius A*.

"No matter how long we run the simulations to let them settle down, most of the simulations still failed that test," Chan said. "They don't quite match the reality, but I think this is more exciting than if everything simply worked out. Now, we can learn some new physics and understand our own black hole better."

The UArizona faculty members working to understand black holes have been tackling this challenge for decades and were part of the research groups that identified the black hole at the center of the Milky Way and the one at the center of Messier 87 galaxy as ideal targets of study. The university also contributed two of the eight telescopes in the EHT array used to create these images – the Sub-Millimeter Telescope on Mount Graham in Arizona and the South Pole Telescope in Antarctica. In 2019, UArizona also added the 12-meter telescope on Kitt Peak in Arizona to the array.

In all, 36 UArizona researchers, graduate students and undergraduate students are involved in the EHT Collaboration, including professors of astronomy Dimitrios Psaltis, Feryal Özel, Dan Marrone and research professor and astronomer Remo Tilanus. Astronomy department head Buell Jannuzi serves on the EHT board.