2026-11-02 –, Conference Foyer 1
Astronomical Projects with Multiple Repositories: A Portal for Consolidated Information and Kubernetes-Based Operations
Authors:
E. Buchholz<sup id="fnref:1">1</sup>, S. Borra<sup id="fnref:1">1</sup>, M. Drobek<sup id="fnref:1">1</sup>, L. Haupt<sup id="fnref:1">1</sup>, H. Heßling<sup id="fnref:1">1</sup>
Affiliation:
<sup id="fnref:1">1</sup> German Center for Astrophysics (DZA), Görlitz, Germany.
Date:
July 30, 2026
Abstract
The antennas and telescopes of the Square Kilometre Array Observatory (SKAO) will produce enormous data streams, making long-term storage of the raw telescope data impractical. Instead, projects such as ‘Transients and Pulsars with MeerKAT’ (TRAPUM) process the data streams in real time. They search for pulsars and transients, storing only data of scientific relevance. This real time processing is performed by a largely automated pipeline, though final steps still require manual inspection.
The Machine-Learning based Pipeline for Pulsar Analysis (ML-PPA) project aims to eliminate the manual steps by providing a fully automated analysis pipeline for pulsar searches, developed by an interdisciplinary team of astronomers, physicists and data scientists.
As the ML-PPA project continues to grow, the increasing involvement of researchers and the expansion of software have introduced challenges in maintaining a well-organised and scalable software structure.
Such project needs to adopt modern engineering practices, including a Continuous Integration and Continuous Deployment (CI/CD) pipeline for each repository, test coverage and automated deployment of documentation. In addition to eliminating isolated information silos within the infrastructure, a single portal was set up to centralise all information.
This presentation provides an overview of the software architecture of the ML-PPA project and explains how the portal and documentation for each of the sub-projects are automatically generated from the source code and made available online. In addition, containerisation and a Kubernetes infrastructure are used to operate the entire ML-PPA system. Both the repositories and the documentation remain in-house, which enables automated updates, thereby reducing maintenance costs in particular.
Future additions to ML-PPA will follow this workflow to ensure high-quality, well-documented code that is seamlessly deployed on a modern infrastructure. The already substantial volume of software documentation makes it difficult to extract relevant information. Therefore, LLMs (Large Language Models) are to be integrated in future to facilitate the use of ML-PPA.
Hold a Master’s degree in Computer Science.
Working for German Center for Astrophysics as a Data Scientist.