The ASKAP data processing pipeline is developed in a modern Dev/Ops framework and is in general fit for the current purpose. But many of the tasks - such as regression testing and deployment are performed manually. Although these processes are aided by considerable scripting resources and have a moderately static configuration, it still takes too much time and manpower to test and release new versions of the processing package. It is often the case that new requested functionality is developed quickly, but released slowly. This has the effect of greatly increasing the time to delivery of new features. The development team reached out to ADACS to perform an audit of their current system of testing and releasing new features to identify ways in which the process can be better automated so that less human interaction is needed to approve the newly developed features. The ultimate goal is to be able to test and release new features at the same cadence as they are able to be developed by the software team.
During the 2025A semester, ADACS engaged with the ASKAP software developers to map out the path from new features being developed and unit tested, to having these features tested as part of the larger ASKAP workflow and passing the final science verification. As expected there were many different technologies used throughout the test and release processes, and no one person had custody of the entire system. After many discussions, much code review, and proof of concept workflows, ADACS was able to make many recommendations that should result in a more robust and automated test and release process, while requiring much less human interaction.
Check out some of our other projects.
ADACS collaborated with researchers in Social Network Analysis to improve the performance of their simulation code. This allowed them to tackle research problems that were previously computationally intractable.
TRACET is a web app that uses VOEvent alerts to decide which transient events to observe with the MWA telescope, streamlining the process of rapid-response observations.
The HORMONE simulation code makes use of a novel self-gravity solver. ADACS implemented MPI parallelisation into the code, enabling it to scale across multiple compute nodes, facilitating larger and more detailed simulations.