Lars Lindberg Christensen | NOIRLab, USA

Fig 1: This NASA/ESA/CSA James Webb Space Telescope MIRI image of N79 in the Large Magellanic Cloud shows a region of interstellar atomic hydrogen that is ionized. Credit: ESA/Webb, NASA & CSA, O. Nayak, M. Meixner.

The Universe, a canvas of celestial wonders, has captivated human imagination for millennia. Advances in astronomical technology, particularly in color imaging from telescopes on the ground and in space, have served to transform our understanding of the Universe and foster a profound sense of connection to the cosmos.

When we stargaze outside under a clear sky with lay people today, the most frequent question is: “Is there life out there?” It is more frequent than the questions about black holes, Mars and lunar landings that were common 30 or 40 years ago. I believe this is a result of our worldview slowly changing.

The answer to the question about life is, of course, tricky. On the one hand, we know that there are billions of worlds out there and likely life as well, but on the other hand it took a long string of seemingly improbable events to produce life. Among them, a mature galaxy billions of years into its life span forming a mature Solar System with sufficient metals to form a rocky planet where water and life appeared. And later, a Moon creating tides, enabling the right conditions for life to emerge from the oceans. And then the meteorite impact some 65 million years ago which altered animal life on the Earth profoundly, as the dominant reptiles were largely replaced by mammals…

And now we are gathered here in the Vatican, with JWST and other telescopes preparing the canvas for the most exciting time of scientific progress! Very fitting to be reflecting in this place.

Over the last 25 years, I’ve had the privilege to work with a talented crew of image processing experts in Europe and the US, like my co-author Mahdi Zamani, working on data from the NASA/ESA/CSA Hubble Space Telescope, the European Southern Observatory, the NSF NOIRLab, the NSF National Solar Observatory, NASA/ESA/CSA James Webb Space Telescope and more.

The journey from monochrome to vibrant, high-resolution color images has been a pivotal element in broadening our cosmic perspective and redefining our place in the grand tapestry of the cosmos. To highlight some examples of what I find to be some of the most pivotal visuals, at least from western science, through history:

1. In 1543 De Revolutionibus by Copernicus showed the Sun at the center of the Solar System and the planets orbiting around it.
2. In 1572 Tycho Brahe’s bright supernova in Cassiopeia showed that the heavens are changeable and nudged a revolution in astronomy (Fig. 2).
3. In 1613 Galileo Galilei’s observations of sunspots showed that the Sun is not perfect, but flawed.
4. In 1785 William Herschel carefully counted the stars in different regions of the night sky and his sister Caroline produced the first drawing of the shape of our star island, the Milky Way.
5. In 1859 Richard Carrington sketched an intensely bright flare on the Sun. This led to what became known as the Carrington Event, the most intense geomagnetic storm in recorded history, that caused fires in telegraph stations. Today it would have resulted in electrical blackouts and widespread damage to satellites and other sensitive systems.
6. In the 1890s pioneering astro-photographer E.E. Barnard imaged the Barnard nebulae, clouds of gas and dust that obscured the more distant stars in the background as he mapped the heavens.
7. In 1919 two expeditions, led by Dyson and Eddington, used a total solar eclipse to measure the position of stars near the Sun so as to verify the minute two-arcsecond bending of light by its gravity — a visual proof of Einstein's mind-bending general theory of relativity.
8. In 1968 astronaut William Anders photographed the Earthrise during the NASA Apollo 8 mission. By some accounts “the most influential environmental photograph ever taken”. It moved our vantage point and worldview from the Earth to space.
9. In 1990 Voyager 1’s Pale Blue Dot took our vantage point even further out.
10. Around 1992 David Malin, known as the Man Who Colored the Stars, was still using photographic emulsion and gave us views of cosmos that moved us even closer to the stars.
11. In 2015 NASA’s New Horizons spacecraft captured a high-resolution color view of the dwarf planet Pluto, exploding our perception of the diversity of worlds in the outer parts of the Solar System.

Fig 2: Star map of the constellation Cassiopeia showing the position (labeled I) of the supernova of 1572; from Tycho Brahe’s De Nova Stella (On the New Star) of 1573. Credit: Tycho Brahe.

Of course, the observations and the scientific papers in many ways speak for themselves but there’s a lot going on under the hood when it comes to visualizing the data in color and removing all instrumental artifacts. Our decades of experience in creating color images from data from telescopes in space and on the ground has given us our significant expertise – however, with the forthcoming data deluge from Rubin and JWST, we are entering a new era in data visualization.

The process involves some design choices – and because of that we have developed rigorous scientific standards for generating ethically correct images for the public.

Fig. 3: One of the first big Hubble mosaics done with 15 HST/WFPC2 pointings of the Crab Nebula. Credit: NASA, ESA, A. Loll/J. Hester (Arizona State University)

It is almost possible to make a recipe for the six steps, or components, that go into these images:

1. Resolution – not just angular resolution, but Photogenic Resolution that we defined in 2013 – field of view divided by angular resolution. Large ground-based imagers can also make amazing images if the number of resolution elements is large! A good example is one of the first big Hubble mosaics – 15 pointings by the WPFC2 of the Crab Nebula that we put together in 2005 (Fig. 3).
2. To get definition and contrast the compression of the immense dynamic range in the astronomical FITS files is, in many ways, the secret sauce. After the astronomical processing the stretch function brings the many nuances into view, before compositing and post-processing. This work is performed in our own interactive tool called the FITS Liberator, now used by most image processing experts. The inverse hyperbolic sine is our go-to function and is flexible enough to handle most objects.
3. The colors are usually assigned as close to the astronomical filters as possible, but for JWST we use chromatic ordering where the least infrared filter is assigned blue and the most infrared is red.
4. Composition, of course, plays a huge role. Books have been written about framing. The classical example is Jeff Hester’s Eagle Nebula from 1995 with Hubble, carefully prepared to work with WFPC2’s bat shape.
5. Signal-to-Noise can be improved in a number of ways – exposure time, good weather, large pixels etc. but also in post-processing through sophisticated use of modern noise-reduction and sharpening tools in frequency space.
6. Experience has taught us that lay people need images that are pristine depictions of space. We invest a significant amount of manual labor into removing artifacts from the telescope and detector – sometimes hundreds of hours for gigapixel images. A case in point are the diffraction spikes – artifacts from the secondary mirror mount that are almost impossible to remove in the cosmetic cleaning and which have somewhat incorrectly become the four-spiked stars we see in popular culture. Maybe we will start seeing more six-spiked stars now, inspired by JWST images.

The full overview of our image processing pipeline is shown below. No need to go into the details but in recent years especially the post-processing and artificial intelligence tools have led to significant improvements and the ability to cope with gigapixel-sized images.

Fig 4. Pipeline for Post-processing of EPO Images (detailed). Credit: M. Zamani & L.L. Christensen.

In conclusion, we have evolved from an understanding that we are the center of the Universe, to the Sun being the center of the Universe, to having no center of the Universe – and are now potentially on the cusp of discovering other worlds with life. And when we do, our worldview will change profoundly again. All eyes will pivot to that world. For a time at least, that may become the center of our Universe.

There is no doubt these are exciting times, and images are arguably the best way to share these momentous discoveries with the public. The huge datasets from JWST and soon Rubin, translated into color images, do not only provide astronomers with invaluable information enabling us to grasp the intricacies of cosmic phenomena. Astronomical images also kindle our curiosity and sense of wonder and inspire a deep appreciation for the vastness and complexity of the Universe – as long as we uphold strict scientific standards and ethical production following the recipe laid out.