Optimising the SAMI cubing code

CI: – Dr. N. Scott

The SAMI data reduction pipeline processes raw data from the SAMI and upcoming Hector instruments, producing thousands of science-quality spectroscopic data cubes that become publicly available data. The current implementation of the ‘cubing’ element of the pipeline suffers from being inefficient and using an algorithm that introduces problematic artifacts into the data. These issues inhibit the use of SAMI and Hector data for a subset of potentially high-impact science, as well as delaying the release of new data to the science teams and the broader community.
The SAMI team have prototyped two alternative cubing approaches (based on Liu et al. (2020) and work by Taylor, Scalzo et al. in collaboration with the SAMI team) which, when properly implemented, should mitigate these issues. This proposal aims to:
– Implement these prototyped cubing algorithms into the existing SAMI reduction pipeline
– Optimise these algorithms and related aspects of the cubing code to facilitate rapid data turn-around
Expected outcomes are:
– An end-to-end pipeline that can produce science-quality data cubes using any of the three (one existing and two proposed) cubing approaches. This will use the existing SAMI data reduction pipeline architecture, allowing selection of the different cubing approaches within the ‘SAMI Manager’ framework.
– A factor of 3 to 10 improvement in the processing time of data cubes. This will allow re-reduction of the large anticipated Hector dataset on the timescale of several weeks, as opposed to several months, as well as rapid reduction of data during observations resulting in more agile observing procedures.
The SAMI reduction pipeline is implemented in Python (https://github.com/SAMI-Galaxy-Survey/sami), with some elements utilising compiled C code. The SAMI team lacks expertise in benchmarking and optimising software, and the experience of ADACS developers in this area would be the key contribution to the project.
We anticipate several future proposals to ADACS to improve other aspects of the SAMI/Hector data reduction pipeline in upcoming proposal rounds. These aspects will benefit significantly from the availability of on-sky data from the upcoming Hector instrument, so we anticipate submitting future proposals when this data becomes available from mid-2021. While we have a strong preference for the above program, we would also consider a smaller allocation corresponding to a detailed code review, with the expectation that this would facilitate a larger proposal in the following semester.