Asynchronous Web Application for Large Sky Surveys Visualisation

Published: July 13, 2023

2023A

Visualising survey images and catalogue photometry is a common challenge for modern large sky surveys with large data volumes. This becomes harder when comparing multiple surveys against each other in various filters, epochs or event carriers (e.g. gravitational waves). Doing these tasks using the traditional astronomical software toolkit could be complicated and inefficient, as it may involve downloading and mosaicking survey images, retrieving and processing catalogue photometry, before finally viewing the data using SAOIMage DS9 (Joye and Mandel, 2003) or making plots using Astropy (Price-Whelan et al. 2022) and Matplotlib (Hunter, 2007).

In preparing for their survey, the 4Most Hemisphere Survey of the Nearby Universe (4HS) team found they struggled to visualise their data and could not check its quality based on cross-matching against literature catalogues alone. Hence, the team reached out to ADACS for support to develop a web-based sky image viewer that can help them to quickly evaluate their data processing pipelines. This falls within Data Central team’s area of expertise, as Data Central is a cohesive ecosystem that brings together data access and aggregation services, cloud storage, Jupyter notebooks for collaborative research, and team management tools, thus enabling researchers to tailor their data experience.

Data Central proposed a web application that dynamically generates Hierarchical Progressive Survey (HiPS) (Fernique et al. 2015) data on demand. HiPS is a useful file format for visualising large datasets online, allowing efficient streaming of small portions using specialised applications like Aladin (Bonnarel et al. 2000) or Aladin Lite (Boch and Fernique, 2014). As generating HiPS data normally takes about a week for the entire 4HS dataset, this Data Central-hosted web application enables users to request a field of interest to be converted to HiPS. Through multiple HiPSgen instances, each of these processes takes only about 10 seconds, resulting in a lean and timely means to provide access to a large sky survey without introducing lengthy delays whenever newer versions may be updated.

The web application was developed to retrieve and visualise information on an individual object from multiple separate data sources. Users can upload target names or coordinates, individually or in a list, to be viewed in a navigable pane. The features of the web application include:

This web application allowed the 4HS team to spot and fix survey issues that had not yet been encountered. For instance, through this app, it was discovered that the orientation of the ellipses from the catalogue photometry was incorrect by almost 90 degree for small parts of the sky.

When asked about the significance of this project, Professor Taylor as the Principal Investigator shared, “As part of our project to map galaxies across the southern sky, ADACS has developed a web-based tool to see and interact with our new maps. This was critical for us: with many terabytes of imaging and catalogues, we cannot hold it on an ordinary laptop, much less ship hard drives back and further! The tool we have developed allows us - and our broader team - to query or retrieve data for particular areas or objects from any browser anywhere. It has helped us to understand the quality and limitations of our data, and so to direct our ongoing efforts to improve things. In the near future, this tool will also form a crucial part of our public data releases, to give astronomers and the wider communities an instant view of galaxies across the southern hemisphere.”

Overall, the asynchronous capabilities of the application, coupled with the dynamic HiPS generation and catalogue retrieval from multiple sources unlocked new capabilities for astronomical visualisation applications that would otherwise be limited by disk space.

Project Details

Node: Macquarie University
Project Length: 9 weeks
Development Team:
  • Simon O'Toole
  • Brent Miszalski
Research Science Team:
  • Edward Taylor
  • Michelle Cluver
  • Thomas Jarrett
  • John Lucey

Check out some of our other projects.

See all projects.