BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.adass.org//adass2026//speaker//SXRUKV
BEGIN:VTIMEZONE
TZID:AWST
BEGIN:STANDARD
DTSTART:20000101T000000
RRULE:FREQ=YEARLY;BYMONTH=1;UNTIL=20051231T160000Z
TZNAME:AWST
TZOFFSETFROM:+0800
TZOFFSETTO:+0800
END:STANDARD
BEGIN:STANDARD
DTSTART:20070325T040000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
TZNAME:AWST
TZOFFSETFROM:+0900
TZOFFSETTO:+0800
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20071028T030000
RRULE:FREQ=YEARLY;BYDAY=4SU;BYMONTH=10;UNTIL=20081025T190000Z
TZNAME:AWDT
TZOFFSETFROM:+0800
TZOFFSETTO:+0900
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:pretalx-adass2026-RCLJJB@pretalx.adass.org
DTSTART;TZID=AWST:20261104T110000
DTEND;TZID=AWST:20261104T111500
DESCRIPTION:The Square Kilometre Array Observatory (SKAO) is constructing t
 he world's largest and most sensitive radio observatory\, with SKA-Low in 
 Australia and SKA-Mid in South Africa. The SKA telescopes will produce dat
 a volumes that require reliable processing pipelines capable of transformi
 ng raw visibility data into science-ready products such as continuum and s
 pectral line image cubes and mechanisms to make the large data accessible 
 to users across the world. While real-time processing supports telescope o
 perations during observations and formation of coherent tied array beams\,
  the computationally intensive calibration and imaging workflows are perfo
 rmed as “batch pipelines” after data acquisition. These workflows comp
 rise multiple stages—including pre-processing\, self-calibration\, conti
 nuum imaging and spectral imaging—that must be orchestrated across heter
 ogeneous compute and storage resources while maintaining reproducibility\,
  robustness and operational efficiency.\nThis paper focuses on the design 
 challenges and emerging architecture of the SKAO Science Data Processor (S
 DP) batch pipeline framework\, developed to support reliable\, observatory
 -scale scientific workflows in the petabyte era.. The framework is intende
 d to support a diverse range of observing modes through modular\, configur
 able workflows that separate scientific processing from execution infrastr
 ucture. The SKAO is constructing the Low and Mid telescopes in stages call
 ed Array Assemblies (AA). The major array assembly stages are labelled as 
 AA2\, AA* and AA4. A key design objective is to provide an architecture th
 at scales across successive processing capabilities as the observatory gro
 ws from AA2  through AA* to the full AA4. As computational and domain comp
 lexity increase across these capability levels\, efficient management of v
 isibility data becomes as important as the execution of the scientific alg
 orithms themselves. \nThese requirements have driven several key architect
 ural decisions. An example is the adoption of the Measurement Set version 
 4 (MSv4) data model\, developed in collaboration with the National Radio A
 stronomy Observatory (NRAO)\, that provides a data representation designed
  for parallel access to large visibility datasets. Together with workflow 
 decomposition\, configuration-driven execution\, and well-defined interfac
 es between pipeline stages\, these design choices establish a robust and e
 xtensible foundation for reliable batch processing while allowing calibrat
 ion and imaging algorithms to evolve independently of the execution framew
 ork. Beyond addressing the immediate requirements of the SKAO\, the propos
 ed architecture establishes a foundation for reliable\, modular and scalab
 le batch processing that can evolve with the needs of future data-intensiv
 e astronomical observatories.
DTSTAMP:20261001T101753Z
LOCATION:Banquet Hall
SUMMARY:Towards Reliable Batch Data Processing for the SKA Science Data Pro
 cessor - Ruta Kale
URL:https://pretalx.adass.org/adass2026/talk/RCLJJB/
END:VEVENT
END:VCALENDAR