With a 10-year, $129 million strategic partnership with the European Southern Observatory (ESO), Australian astronomers have had great success at securing time on ESO instruments, culminating with the acceptance of two ESO large programmes: GECKOS and MAUVE. Large programmes are observing programmes of a minimum of 100 hours of telescope time that are treated with a higher priority over normal programmes, and typically only 1-3 are awarded per instrument each semester.
As large programmes have the potential to lead to a major science breakthrough, their technical challenges are also significant. The GECKOS and MAUVE teams reported that the data flow from the telescope to the user was the main issue as it was extremely labour- intensive. Prior to this ADACS project, the data retrieval process required a user to:
Additionally, the bash download script that ESO provides does not include a checksum, making it impossible to tell whether the files are downloaded correctly. Finally, it appears that ESO is throttling download speeds such that files download at 6-12 Mbps, even when internet speeds are in the 100s of Mbps. PIs would frequently spend a whole morning just to download the data as ESO typically sends an email to program PIs at 11pm AEDT to announce when new data become available.
The eso-downloader Python package developed in this ADACS project provides a simple CLI interface to the ESO archive to download entire proposals from the ESO archives. It also provides Pythonic wrappers around each of the components of the download process: getting the token to access proprietary data, finding the OBs for a proposal, finding the science frames for an OB, getting the calibrations for a science frame, and checking the quality of an OB, allowing for an automated download. The Python package is now available on PyPI, and community contributions are welcomed at the eso-downloader repository.
Check out some of our other projects.
Converting a matlab script into a python library, so that IMOS researchers can provide their acoustic data in a more widely accessible data format.
ADACS collaborated with researchers in Social Network Analysis to improve the performance of their simulation code. This allowed them to tackle research problems that were previously computationally intractable.
ADACS developers collaborated with the Parallel Bilby team to overhaul their code, improving its structure, establishing tests for reproducibility, and instilling good development practices. This work set the foundations for future improvements to th