Early operations with the LOFAR processing framework

Abstract

The LOw Frequency ARray (LOFAR) telescope is undergoing a major upgrade which will re-position it in the next decade among the most data intensive astronomical facilities. Therefore, the existing data infrastructure is evolving to become cost-effective and to enable the production of science level data that is delivered directly to the community. Sustainable science operations will rely on a robust data processing framework distributed on several data centers forming the backbone of the LOFAR science archive, automated quality assurance processes, as well as on efficient computing pipelines. To build and gain experience with operating such a complex system a pathfinder has been defined, the LOFAR Data Valorisation (LDV) project. With this contribution the current status of developments of a scalable, distributed data processing framework will be discussed, as well as the testing of the implemented workflows to generate Advanced Data Products (ADPs). Finally, an overview of the achieved operational results for the last two years will be given.