Central shifts in the ALICE Run Control Centre
Having a strong team of motivated and knowledgeable physicists and engineers (or experts) is vital when it comes to operating an experiment as complex as ALICE. A part of the well-established hierarchy in the ALICE Run Control Centre are the four central shifts – Experiment Control System (ECS), Detector Control System (DCS), Data Quality Monitoring (DQM), and Shift Leader (SL). In four consecutive issues of ALICE Matters we will present the central shifts through the eyes of the shifters and the run coordinators of the systems they are responsible for. We begin with the ECS shift.
The Experiment Control System (ECS) is the top control level of the ALICE experiment. It coordinates the operations controlled by three other systems. First, the Data Acquisition System (DAQ), which handles the data coming from the detectors and sends it to the Grid. Second, the Central Trigger Processor (CTP), which takes all the trigger inputs and generates the trigger outputs. Third, the High Level Trigger (HLT), which takes the huge amount of data and reduces it in order to fit the available storage bandwidth by preserving only the physics information of interest.
Since those four systems are tightly connected, they fall under the responsibility of the ALICE DAQ group and there is only one ECS/DAQ/CTP/HLT shifter, making sure they operate properly. “Having the person who takes care of the ECS also taking care of the three systems underneath it is quite natural. If a problem occurs with one of them, he has to find a way to solve it. If he can't, he contacts one of the three people on-call” – says Roberto Divi?, system run coordinator of the DAQ. The shifter is obligated to report the current situation to the shift leader or to the run manager and to comply with their instructions. He also has the responsibility of setting up and controlling the operation of 19 ALICE detectors plus six ALICE systems, including the DCS and the DQM. Although they are not under the direct control of the ECS, they are closely interacting with it in order to ensure the correct operation of the whole experiment.
Roberto Divi? is also responsible for preparing the candidates before they enter the Control Centre first as trainees and then as shifters. So far he has done 700 trainings. His class lasts half a day and it has one big advantage – it can be done remotely. “I chose to take the ECS shift in particular because I could do the training via Vidyo while I was still in the USA, where I'm based” – says Deepa Thomas, a postdoc in ALICE. After the course the candidates spend two days as trainees, observing the current shifter's actions and asking him or her questions. On the third day they take care of all the necessary actions and operations, while being under the supervision of the shifter. This is the so called shadow shift. The final step is to take an online test to prove that they are ready. Besides skills and knowledge, the other main thing Roberto is hoping to see in the future shifters is motivation.
For Julien Hamon, who started his PhD with ALICE a few months ago, taking the ECS shift was an opportunity to better understand how the experiment works. “I thought it would be hard, but if you are conscientious, you can rapidly gain experience and become independent. Moreover, the shift leader is supervising your work so the probability of making a mistake is quite low. Before the run starts, we have to configure the partition, which means that we define a set of detectors and triggers that will work together during the data taking. Then we monitor the Data Acquisition and make sure that everything is fine. For instance, we look at the event rate and prevent it from going either too high or too low. I had never been in the ARC before I came here for the training. I configured the detector and launched the run for the first time. It was very exciting!”
Nur Hussain, also a PhD student, is also excited about taking this shift, but for a different reason. “I get really enlightened while we have physics runs. Thinking that later on we will be analysing those same data, sitting in different parts of the world, makes me happy. I feel very proud about being a part of the ALICE team.”