Software development for the COMPASS experiment
Upcoming SlideShare
Loading in...5
×
 

Software development for the COMPASS experiment

on

  • 210 views

38th Software Development, Ostrava 2012

38th Software Development, Ostrava 2012

Statistics

Views

Total Views
210
Views on SlideShare
210
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Software development for the COMPASS experiment Software development for the COMPASS experiment Presentation Transcript

    • Software development for the C OMPASS experiment Martin Bodlák1 Vladimír Jarý1∗ Igor Konorov2 Alexander Mann2 Josef Nový1 Stephan Paul2 Miroslav Virius1 1 Faculty of Nuclear Sciences and Physical Engineering Czech Technical University in Prague 2 Physik-Department Technische Universität München Conference “Tvorba softwaru 2012” 24th May 2012, Ostrava∗ Vladimir.Jary@cern.ch Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Overview 1 Introduction C OMPASS experiment 2 Current DAQ system Architecture of the system DATE package 3 Control and monitoring software for a new DAQ system Motivation and requirements Overview of the hardware architecture Layers of the DAQ software Implementation details Performance tests 4 Conclusion and outlook Vladimí Jarý et al. Software development for the C OMPASS experiment
    • COMPASS experiment COMPASS: Common muon and proton apparatus for structure and spectroscopy experiment with a fixed target situated on the Super Proton Synchrotron particle accelerator at CERN, [1] scientific program approved in 1997 by CERN experiments with hadron beam (glueballs, Primakoff scattering, charmed hadrons,. . . ) experiments with muon beam (gluon contribution to the nucleon spin, transverse spin structure of nucleons,. . . ) multiple types of polarized target data taking started in 2002 plans at least until 2016 as COMPASS-II 3 programs: GPDs, Drell-Yan, Primakoff scattering international project: cca 250 physicists from 11 countries and 29 institutions Vladimí Jarý et al. Software development for the C OMPASS experiment
    • COMPASS spectrometer polarized target on the left, length approximately 50 m COMPASS spectrometer, image taken from [1] spectrometer consists of detectors: 1 measurement of deposited energy (calorimeters) 2 particle identification (RICH, muon filters) 3 particle tracking (wire chambers) Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Terminology event: collection of data describing flight and interactions of particle through the spectrometer roles of the data acquisition system (DAQ): 1 reads data produced by detectors (readout) 2 assembles full events from fragment (event building) 3 sends events into a permanent storage (data logging) 4 enables configuration, control, and monitoring (run control) 5 preprocesses and filters data (e.g. track reconstruction, online filter) trigger : selects physically interesting events (or refuses noninteresting events) in a high rate environment with minimal latency trigger efficiency : = Ngood(selected) /Ngood(produced) < 1 DAQ deadtime: D = timesystem is busy /timetotal when system is busy, it cannot accept any other trigger which leads to loss of data Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Overview of the TDAQ system Structure of the trigger and data acquisition system according to [4] Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Current DAQ architecture influenced by the cycle of the SPS particle accelerator: 12 s of accelaration, 4.8 s of extraction (spill/burst) key aspects: multiple layers, parelelism, buffering 1 detector (frontend) electronics: preamplify, digitize data 250000 data channels 2 concentrator modules (CATCH, GeSiCA): perform readout (triggered by the Trigger Control System) append subevent header 3 readout buffers: buffering subevents in spillbuffer PCI cards makes use of the SPS cycle to reduce data rate to 1/3 of the onspill rate, roughly stable data rate on the output (derandomization) 4 event builders: assemble full events from subevents send full events to the permanent storage, store metainformation about events into the Oracle DB additional tasks (online filter, data quality monitoring) Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Current DAQ software based on the ALICE DATE package[2] DATE distinguishes two kinds of processors: 1 local data concentrators (LDCs) perform readout of subevents, correspond to readout buffers 2 global data collectors (GDCs) perform event building, correspond to event builders requirements on the nodes: 1 all nodes must be x86 compatible 2 all nodes must be powered by GNU/Linux OS 3 all nodes must be connected to the network supporting the TCP/IP stack flexible system (fixed targer mode × collider mode) scalable system (full scale LHC experiment × small laboratory system with one processor) performance: 40 GB/s readout 2.5 GB/s event building 1.25 GB/s storage Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Functionality DATE provides: 1 readout, data flow 2 event building 3 run control 4 interactive configuration (based on the MySQL database) 5 event monitoring (COOOL) 6 data quality monitoring (MurphyTV ) 7 information reporting (infoLogger, infoBrowser ) 8 online filter (Cinderella) 9 load balancing (EDM, optional) 10 log book 11 ... Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Problems with existing DAQ system Motivation 260 TB recorded during the 2002 Run, 508 TB during the 2004 Run, more than 2 PB during the 2010 Run increasing number of detectors and detector channels, trigger rate ⇒ increasing data rates aging of the hardware ⇒ increasing failure rate of hardware PCI technology deprecated Main idea of the new system replace ROBs and EVBs by custom FPGA-based HW hardware based data flow control and event building smaller number of components, higher reliability Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Overview of the hardware architecture frontend electronics and concentrator modules unchanged readout buffers and event builders replaced with custom hardware: Field Programmable Gate Array (FPGA) technology FPGA card designed as a module for Advanced Telecommunications Computing Architecture (ATCA) carrier card, in total 8 carrier cards: 6 for data multiplexing 2 for event building each carrier card equipped with 4 FPGA modules different functionality, same firmware FPGA card equipped with 4 GB of RAM, 16 serial links (bandwidth 3.25 GB/s) softcore processor on cards for powering control and monitoring software, communication based on Ethernet ROBs and EVBs will be used for computing farm Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Hardware architecture Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Requirements analysis Requirements: distributed system, communication based on TCP/IP compatibility with Detector Control System compatibility with software for physical analysis remote control and monitoring multiple user roles real time not required Decisions: use the DIM library for communication do not use the DATE package possibly reuse some DATE components (COOOL, MurphyTV ) keep data format unchanged Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Software architecture Roles participating in the control and monitoring software Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Roles participating in the software 1 Master process controls slave processes receives commands from GUI authenticate and authorize users reads and writes configuration to online database 2 Slave processes monitor and control the hardware receive configuration information and commands from the master process 3 GUI receives information about health of the system from the master process sends commands to the master process that distributes these commands to the slave processes 4 Message logger: collects messages produced by other processes and stores them into database 5 Message browser: displays the messages produced by other process Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Implementation details communication between nodes based on the DIM library implementation in Qt framework slave processes implemented in C++ language, without Qt scripting in Python (e.g. starting of the slave processes) MySQL database (compatibility with Detector Control System and DATE) complex system ⇒ describe behavior of the master and the slave processes by state machines Vladimí Jarý et al. Software development for the C OMPASS experiment
    • State machines State machine describing behavior of the master process Vladimí Jarý et al. Software development for the C OMPASS experiment
    • DIM Library[3] developed for the DELPHI experiment at CERN asynchronous one-to-many communication in heterogenous network environment [3] based on the TCP/IP interfaces to C, C++, Python, Java languages communication between servers (publishers) and clients (subscribers) through DIM Name Server (DNS) types of messages: services updated at regular intervals services updated on demand commands Vladimí Jarý et al. Software development for the C OMPASS experiment
    • DIM Name Server Position of the DIM Name Server Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Evaluation of the system Test scenario: number of nodes: 2 - 16 message size: 100 B - 500 kB C OMPASS internal network during winter shutdown (Gigabit Ethernet) standard x86 compatible hardware (event builders) Tests performed: performance is system able to update information about status of hardware every 100 ms? stability Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Results of the performance tests Transfer speed as a function of size of the message Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Results of the stability tests Stability of the software in time Vladimí Jarý et al. Software development for the C OMPASS experiment
    • Summary and outlook 1 Analysis of the existing data acquisition system based on the DATE package scalability issues, deprecated technologies (PCI bus) 2 Development of control and monitoring software for new DAQ architecture analysis of requirements on software description of the hardware architecture definition of roles and behavior of the system implementation performance tests 3 Goals: to test system on the real hardware to have fully functional system in 2013 to deploy the system in 2014 Vladimí Jarý et al. Software development for the C OMPASS experiment
    • The bibliography P. Abbon et al.: The COMPASS experiment at CERN, In: Nucl. Instrum. Methods Phys. Res., A 577, 3 (2007) pp. 455–518. See also the COMPASS homepage at http://wwwcompass.cern.ch T. Anticic et al. (ALICE DAQ Project): ALICE DAQ and ECS User’s Guide, CERN EDMS 616039, January 2006 C. Gaspar: Distributed Information Management System [online]. 2011. Available at: http://dim.web.cern.ch W. Vandeli: Introduction to Data Acquisition, In: Internation School of Trigger and Data Acquisition, Roma, February 2011 Acknowledgement This work has been supported by the MŠMT grants LA08015 and SGS 11/167. Vladimí Jarý et al. Software development for the C OMPASS experiment