Accelerating Astrophysical Insights: A DiFX Project – Bailey Sykes

I undertook my ADACS internship part-way through the third year of my PhD. Having worked with simulations and data throughout my time in astrophysics research, I came into the program confident around computers and code; however, I felt like my skills were domain-specific. I was also almost entirely self-taught and pretty sure that I was not following best practices. An ADACS internship seemed the perfect opportunity to build a broader software development skillset and learn how to collaborate with a more diverse group.

Starting out, the ADACS team was sure to emphasise that my purpose here was not productivity, but learning and exploration. I was free to explore whatever topics took my interest and try out a few of their new or ongoing projects to see what piqued my interest most. I ended up focusing on two different projects during my time; one was a CI/CD project whose start coincided with the start of my internship, and an ongoing GPU-based computational project. I also briefly looked at the Django web framework in the first few days, and then learnt a bit of the Chapel programming language when it caught my eye later on.

The CI/CD project dealt with the Monte Carlo radiative transfer code MCFOST, developed by the group at Monash University. I had not been exposed to the CI/CD workflow before and it was very interesting being able to follow the process of implementing it from start to finish. The project involved setting up scripts to run automatically through GitHub Actions to test that the MCFOST code compiles (including in a mix of computing environments), produces correct results, and is acceptably performant. I feel like I was able to contribute meaningfully to developing these features, with the ADACS team helping out where necessary along the way. I was also able to observe the development of an automated release workflow which nicely rounded out the project.

I have experience with MP/MPI code from my prior simulation work, however, I haven’t had the chance to learn any GPU/CUDA programming. I was lucky enough that ADACS had a project working with the DiFX correlator, moving the bulk of the signal processing from the CPU to the GPU. A previous project had achieved a large part of this goal already – including moving the FFT and various signal corrections to the GPU – however, the unpacking of compressed raw data from telescopes into usable data arrays was still done on the CPU and was a performance bottleneck. DiFX was a challenging code to work with since it strives for exceptional performance while covering a large range of modes and data input types. I was able to work both with ADACS and directly with the science team to understand the code and the required changes. Although there were many setbacks along the way, I was able to finish my time at ADACS having contributed a functional – if slightly rough around the edges – GPU unpacking code which improved the data unpack speed by a factor of ~5.

On reflection, my time at ADACS was very enjoyable and fulfilling. Although not required, I felt like I was able to make some significant contributions to the projects I worked on. The ADACS team was very friendly and supportive, making me feel valued and appreciated. I anticipate that the skills I have gained through this experience will be valuable wherever I end up working in the future.