The SRCNet Federated Execution Framework: Bringing Scientific Workflows to the Data
2026-11-03 –, Conference Foyer 1

The SKA Regional Centre Network (SRCNet) will support scientific analysis through a federation of independently operated computing and storage facilities. Executing workflows across this infrastructure requires a common interface that accounts for data location, heterogeneous resources, and local scheduling policies.

We present the SRCNet Federated Execution Framework, a prototype that connects scientific workflow submission to execution on participating sites. A Computing Broker identifies eligible sites using resource capabilities and data locality.

Site-level Job Gateways provision pilot workers through local Kubernetes or Slurm infrastructure. HTCondor then matches queued workflow tasks to compatible pilots once resources become available, a mechanism known as late binding. This decouples task submission from worker allocation while allowing each site to retain control over its resources and scheduling policies.

The poster explains the architecture and follows a workflow from submission through site eligibility checks, pilot provisioning, execution, and status reporting. We illustrate the approach with the ICAL/Rapthor radio-interferometric calibration and imaging workflow, drawing on the prototype’s multi-node execution demonstration. We also discuss integration challenges involving authentication, delegated credentials, data access, and the boundary between federated services and site operations.

We conclude with lessons from building the prototype and priorities for operational deployment, including broader site validation, robust data staging, and execution identity management. The framework offers a concrete example of how existing scheduling technologies can support a common scientific execution service across heterogeneous astronomical computing facilities.


This poster presents collaborative development within the SRCNet Execution Framework team, with Kim Hoogenberg and Dev Null as presenters.

After being graduated in 2003, Kim has worked in Amsterdam for a few years working on realtime news and rates distribution and "Over The Counter" trade engines. After that period he worked as an independent Software Engineer for almost 20 years for multiple companies and governmental organisations. Now back to his roots, he is living in Drenthe and working in what could be called his backyard at ASTRON as a Senior Software Engineer with team Rainbow where he is able to combine his passion of the Universe with his profession. Since 2026 Kim also joined SRCNet where he is working on the Federate Execution Platform for the SKA telescope. When he's off duty, Kim loves to do sports and cooking and he is happily living with his wife and 5 kids in the small village of Schoonoord.

This speaker also appears in: