Fast Control Latency Uncertainty Elimination for BESIII ETOF Upgrade* Abstract: Key words PACS: 1 Introduction

Transcription

1 Fast Control Latency Uncertainty Elimination for BESIII ETOF Upgrade * Yun Wang( 汪昀 ) 1, 2, Ping Cao ( 曹平 ) 1, 2;1), Shu-bin Liu ( 刘树彬 ) 1, 3, i An ( 安琪 ) 1, 2 1 State Key Laboratory of Particle etection and Electronics, University of Science and Technology of China, Hefei, , China 2 School of Nuclear Science and Technology, University of Science and Technology of China, Hefei , China 3 epartment of Modern Physics, University of Science and Technology of China, Hefei , China Abstract: In Beijing Spectrometer (BES III) end-cap time of flight (ETOF) upgrade, to fan out fast control signals precisely to new ETOF electronics, new fanning topology is proposed based on the existing architecture. However, uncertainty of transfer latency is introduced by new fanning channel, which will degrade the precision of fast control. In this paper, latency uncertainty elimination for BESIII ETOF upgrade is introduced. The latency uncertainty is determined by a TC (time-digital-converter) embedded in FPGA and is eliminated by re-captured at synchronous and determinate time. Compared with the existing method of barrel-cap TOF (BTOF), it has advantages of flexible structure, easy for calibration and good adaptability. Field tests on BES III ETOF system shows that this method can eliminate transfer latency uncertainty effectively. Key words: Transfer latency uncertainty, Seres, BES III, Endcap TOF PACS: Ca 1 Introduction The Beijing Electron Positron Collider (BEPC) and the Beijing Spectrometer (BES) were upgraded to BEPC II and BES III respectively in the summer of To further improve the overall time resolution of the particle identification (PI), the endcap time of flight (ETOF) is upgraded with newly developed MRPC detectors [1,2,3]. After upgrade, the total time resolution will be improved significantly from 138ps to about 80ps [4]. To achieve this specification, new readout electronics for ETOF should be developed. In the existing BESIII TOF system, there are 2 VMExP crates, inside each of which there is a fast control module (FCTL) receiving signals from the fast control system (FCS). FCS is the control kernel for BESIII trigger system, which is comprised of trigger timing controller (TTC) and related interfaces. TTC can generate and fast control signals synchronized with the beam collision strictly. According to the timing of trigger, TTC fans these synchronous signals to each readout electronic sub-system and receives status data back from these electronics. For new ETOF electronics, two extra new VME crates are added in order to adapt the large amount of MRPC channels. So there are 4 VME crates for TOF sub-system totally. However, there are only two critical and calibrated channels between FCS and the TOF sub-system for fast control signals fanning out. So new topology needs to be proposed for transmitting fast control signals to new ETOF electronics. The key point for transmitting fast control signals is that ETOF should be synchronized with the barrel TOF (BTOF) system. Besides, whichever fast control signals (ETOF or BTOF) should also be transmitted with determined latency. There are several different methods to transmit fast control signals in particle experiments. In Barbar experiment, fast control signal is transmitted through electric cables without being encoded [ 5 ]. There are no any transmission latency uncertainty in this experiment. In LHC, fast control signal is transmitted through dedicated ASIC chip designed by CERN [6]. The transmission latency uncertainty is eliminated by special technique of encoding and synchronizing code [7]. While in BESIII, fast control signal is distributed with technique of serializing and deserializing (Seres) [8]. Fast control signals are encoded from 8bit to 10bit data with timing information and then serialized into bit stream at the transmitting side for output. However, at the side of receiver, there can be 10 discrete phase when recovering from the bit stream, which leads to latency uncertainty for distributing fast control signal. In BESIII FCS design, a clever method is used to re-synchronize the fast control data at determinate time in order to avoid the area of uncertainty [8, 9 ]. This method achieves success only when the length of distributing channel equals to each other. However, two extra newly brought VME crates change the distributing topology for ETOF fast control signals, which leads to the failure of existing method. In this paper, a new method is presented. It is easy to be used and can be carried out automatically. * Funded by CAS maintenance project for major scientific and technological infrastructure (IHEP-SW-953/2013) 1) cping@ustc.edu.cn 1

2 2 Fast Control Latency Uncertainty in ETOF upgrade 2.1 Structure after upgrade There are several sub-systems such as TOF, Muon, MC, EMC connected to the BESIII FCS [8,10]. Each of them needs receive fast control signals from FCS. FCS residents in the BES III trigger system. Inside of FCS, there are totally five fiber channels, each of which is dedicated for fanning fast control signals (L1 trigger signal, fiber reset signal etc.) out to one sub-detector system. For TOF fast control sub-system (TOF- FCTL), there are 4 electronic modules (TOF- FCTL1~4 in Fig. 1) for receiving signals distributed from FCS after ETOF being upgraded as shown in Fig. 1, two for ETOF and two for BTOF respectively. There are two s on each ETOF fast control module, one is the global () distributed from TOF main, another is the recovered from the bit stream (blue lines in Fig. 1) sent from BTOF. The phase difference between these two s is noted as Δ. distribute fast control signals to all sub-systems. For the purpose of compatibility, the same technique of distributing signal as FCS is used in ETOF module design. The basic principle for using the technique of Seres is that the transmitter sends encoded data according to a reference, while the receiver acquires data according to the recovered from the transmitted bit stream. So the acquired data is aligned to the recovered. In Fig. 2, yellow signal denotes the transmitting reference, while the green one denotes the recovered at the receiving point. Unfortunately, at each time the Seres powered up, there is uncertainty for the phase of recovered as shown in Fig. 2, where the oscilloscope runs in persistence mode. This uncertainty is not a problem for traditional communication applications, where the correctness of data is the key point. However, for BESIII FCS distributing fast control signals, the most urgent point is the time of fast control signal being recognized. Uncertain phase in the recovered will inevitably lead to uncertainty in trigger time. TOF-FCTL2 1 st floor BTOF FCS TOF-FCTL1 3 rd floor BTOF Δ1 TOF-FCTL3 East ETOF TOF main 499.8MHz RF Clock Fig. 2 Phase uncertainty in the recovered. Δ2 TOF-FCTL4 West ETOF Fig. 1 Structure for fast control signal fanning out after ETOF upgraded Theoretically, to guarantee the performance of synchronization, ETOF needs 4 fast control distributing channels for receiving signals from FCS. However, in order not to change the existing FCS structure, there are only two fiber channels (red lines in Fig. 1) for FCS distributing fast control signals to TOF-FCTL, just as the situation before ETOF upgrade. In ETOF electronics, there are two extra new VME crates, inside of which there is a TOF-FCTL module respectively. In Fig. 1, to distribute fast control signals to all TOF- FCTL modules, the module residing in the 3 rd floor BTOF crate acts as a router. It receives fast control signals from FCS and then fans them out to each ETOF module. Two extra channels (blue lines in Fig. 1) are added in the new distributing topology. 2.2 Uncertainty introduced by Seres The existing FCS use TLK1501 [ 11 ] to The reason of uncertainty occurrence for fast control signal is that the transmitted critical data is captured by the uncertain recovered. If it is captured by the stable global synchronous, this uncertainty can be eliminated. So, in the existing BTOF module, the encoded fast control data capturing time is postponed to avoid the unstable area (responding to the uncertain recovered ) in data bus according to the global synchronous, not to the recovered [8,9]. That is the transmitted data is recaptured by global synchronous which is shifted by a LL with configurable delay ability. This can achieve success in the existing TOF subsystem based upon the two facts. One is that there is a global precise network everywhere in BESIII electronics, another is that there are only two same distributing channels (red lines in Fig. 1) from FCS to TOF sub-system, which makes the acquiring time can easily be calibrated and adjusted. uring calibration, FCS and TOF are placed in the same place and both are set into debugging mode. FCS continuously sends trigger data to TOF. At the TOF side, it is convenient to adjust the delay value according to compare the 2

3 transmitted and received trigger signal with an oscilloscope. Unfortunately, case is not so simply after ETOF upgrade. Firstly, distributing channel for ETOF differs from that of BTOF. Secondly, no any BTOF electronic module (except for the TOF- FCTL1 module in Fig. 1) can be modified in order to avoid unpredictable affection to BTOF. Under this restriction, it is not easy to find the time for acquiring trigger data stably and synchronously with the existing method because: 1. FCS and TOF system are both installed in BESIII already. They are away from each other, which makes it impossible to monitor the transmitted and received trigger in one oscilloscope. 2. FCS is in working mode now and cannot be configured back to debugging mode again or moved away from BESIII, even temporally for test. 3. Besides upgrading ETOF sub-system, all other subsystems in BESIII can run normally with good specification. So the BEPCII is still running. It is forbidden to affect any sub-systems on BEPCII during the whole upgrade phase. The accelerator machine time used for new ETOF electronics debugging should be considered as short as possible. 4. Furthermore, even though there s any chances for calibrating with FCS, the existing adjusted parameters in BTOF are not adapt for ETOF because that all new FPGA chip is used for upgrade whose specification of internal logic cell differs from the old BTOF one. The ETOF electronics need to be re-calibrated in BESIII, which will consume accelerator machine time too much. So the existing synchronizing fast control signal method doesn t work for this new situation of ETOF upgrade. ETOF electronics should have the ability of eliminate latency uncertainty automatically in the field of spectrometer. 3 Uncertainty Elimination Because of the uncertainty of recovered () at the receiver side, there is an unstable period in the fast control data which is recovered according to the. This unstable period is 1.2ns less than one half of time as area P or N shown in Fig. 3. The has the same frequency with the global synchronous, so if the recovered data is sampled at both edges of a reference (shifted from the TOF global, abbreviated as ), at least one value is reliable. Fig. 3 shows the timing graph where the rx_data (16-bit width) is aligned with. One of s edge (leading or trailing) must be in the stable area of rx_data. To improve the reliability of data capturing, the middle position (red dash line in Fig. 3) of rx_data stable area can be considered as the ideal capturing position, which can make the capture margin period large 3 enough. Furthermore, this position is determinate and doesn t change whenever Seres powers up. Supposing there is no any skew between and (Δ=0 in Fig. 1), if fixing the position of and then checking that of, there will randomly distribute 10 phases, among which both phase 81 (falling edge) and 261 (rising edge) correspond to this ideal position (bold black lines shown in Fig. 3). (shifted) rx_data T(24ns) stable area 1.2ns ideal capturing position Fig. 3 Timing graph for elimination principle area P area N area PN The problem left is how to obtain the with phase 81 (or 261 ). Actually, this finding process can be considered as checking the relative position between and equivalently. There s no need to exactly distinguish each position among these 10 phases. The principle is to make the real capturing position move to that of ideal as near as possible. So the relative phase difference can be measured by a TC with bin size of 3ns. One cycle (T) can be divided into 8 parts with initial phase value from ϕ 0 to ϕ 7, where ϕ i =i*45. If the TC s dynamic range is only one period cycle (T), it will generate 8 data (0~7) corresponding to each phase respectively (Fig. 4). start ϕ6 stop ϕ7 ϕ0 T (24ns) ϕ1 ϕ2 ϕ3 ϕ4 ϕ5 ϕ6 ϕ7 ϕ0 ϕ1 ϕ Fig. 4 TC data and phase bin size =3ns TC data In Fig. 4, the rising edge of is used to start the TC counting, while to stop it. If the phase difference is less than one bin size (red block in Fig. 4), the TC will give output value of 0. So according to the output code from TC, it is easy to know about the relative position between and. However, because the existence of jitter or bit dis-matching of rx_data (16-bit width), TC may generate two values randomly during the critical period of bin transition (gray block in Fig. 4). For the case of 0 phase difference, TC can generate 0 or 7, while

4 for the case of 90, the data is 1 or 2. position ϕ ϕ+180 (1) (2) (3) Δ1 T (24ns) min 10.8ns max δ Δ2 Δ3 δ Δ3-T TC data Fig. 5 Algorithm for finding rx_data re-capture position Because of the existence of uncertainty in rx_ phase, the phase difference between and can be considered as following: Δϕ=Δ+δ where Δ denotes the difference caused by signal distributing channel (blue lines in Fig. 1), and δ denotes the uncertainty caused by Seres. In Fig. 5, dark grey blocks denote the uncertain area where the rising edge of occurs. The width of this area is 10.8ns. The center of the attached area (green block in Fig. 5) where rx_data is stable is the ideal position for data capturing where the should be shifted to. There are three possibilities for phase difference (Δ) discussed as following: (1) 0 Δ<T/2+1.2 The position of rising edge swings in one period T as shown in Fig. 5. TC generates the following data: {0, 1, 2, 3, 4} when Δ is digitized as 0; {1, 2, 3, 4, 5} when Δ is digitized as 1; {2, 3, 4, 5, 6} when Δ is digitized as 2; {3, 4, 5, 6, 7} when Δ is digitized as 3. (2) T/2+1.2 Δ<T The position of rising edge overflows outside one period T as shown in Fig. 5. TC generates the following data: {4, 5, 6, 7, 8}, when Δ is digitized as 4; {5, 6, 7, 8, 9}, when Δ is digitized as 5; {6, 7, 8, 9, 10}, when Δ is digitized as 6; {7, 8, 9, 10, 11}, when Δ is digitized as 7. Since the TC dynamic range is only one cycle, the swing area (from TC output data) will be folded back to the beginning of this cycle. So the data pair generated by TC is: {4, 5, 6, 7, 0}, when Δ is digitized as 4; {5, 6, 7, 0, 1}, when Δ is digitized as 5; {6, 7, 0, 1, 2}, when Δ is digitized as 6; {7, 0, 1, 2, 3}, when Δ is digitized as 7; Compared with the first situation, the <min, max> value pair is definitely <0, 7> whichever value Δ is. (3) Δ>T This case is simple. It can be classified into case 1 or 2 if the TC counting start signal is switched to the next following rising edge of 4 until (Δ-n*T)<T. According to the above mentioned discussion, the algorithm for re-capturing rx_data at the receiving side is listed as following: (1) Receiver resets Seres periodically to generate with uncertain phase and set rf=0. (2) Measure the phase difference (Δ<T, normalized to one cycle) between and, and count the min and max value to form <min, max> pair as well. (3) If <min, max> pair is not <0, 7>, shift with phase (min* ) and then go to step (5). (4) If <min, max> pair is <0, 7>, shift with phase 180, rf=1 and then go back to step (2). (5) Capture rx_data with the shifted (rf=0) or reversed (rf=1) at its falling edge (red circle in Fig. 6) and then go to step (6). (6) Finish. shifted reversed shifted 0 (1) 180 (2) 45 Δ1 Δ2 T (24ns) min* min* Δ2 * TC data Fig. 6 Selection of capturing rx_data (green blocks denote the stable area for rx_data) at shifted colored with red Fig. 7 is the block diagram of uncertainty elimination algorithm implemented in FPGA. The recovered data (rxd with 16-bit width) is directly fed into a group of flip flops inside one common logic array block in FPGA for data alignment. It is synchronized to. Then the aligned data is sampled by two groups of flipflops simultaneously, one works on phase ϕ, another on ϕ+180 (ϕ is automatically obtained through the phase difference measuring flow and the resolution of phase shifting is 200ps). The two groups of flip-flops must be constrained together to ensure that the 16-bit aligned data are sampled under the same condition. After being synthesized and implemented in FPGA, time analysis shows that the data skew of all the sample FFs is only a few picoseconds. Finally, one of the sampled data is selected as the stable output data according to the above mentioned algorithm. rxd PLL dynamic ϕ +180 phase shift enabled ϕ data 0 PLL 90 for phase 180 measure 270 MUX TC phase monitor min CSRs max VME RW stable data

5 Fig. 7 Logic block diagram for uncertainty elimination The sampling phase can be adjusted by logic automatically. A multi-phase interpolation technique based TC [ 12 ] implemented in FPGA is used to measure the phase difference of and. Obviously, because the phase difference is less than one period (24ns), the TC doesn t need large dynamic range, which can further simplify the TC structure. If the Seres is triggered into the cycle of being enabled and disabled continuously, it will generate with uncertain phase. The phase difference (Δ) can be measured by the phase monitor. 4 Experiments & Verification To verify this proposed method, an experiment is carried out. The test platform is shown in Fig. 6. VME crate A SFP Transceiver UT received control signal test position TLK 1501 process logic FPGA VME Backplane The slave module while the recovered signal (blue line) at the UT side is monitored in infinite presentation mode, which means the uncertainty between the transmitted and recovered signals can be captured by the oscilloscope. Obviously, Fig. 8 shows that the latency between these two signals is fixed. The fast control signal uncertainty is eliminated. Actually, Fig. 8 is captured after this test platform running on 8 hours with Seres being reset over and over, which means this proposed method has good advantage of stability. Furthermore, the upgraded ETOF fast control module is implemented with this method. Tests are also carried out in BESIII environment. Results show that the uncertainty is eliminated. Till now, the new modules are installed into the BESIII upgrade ETOF VME crates (Fig. 10) and work well. VME crate B test signal generating module SFP Transceiver TLK 1501 oscilloscope trigger source Fast control Control signal generation FPGA VME Backplane The main module Analog signal source 500MHz RF Fig. 9 Fast control signal uncertainty monitoring result (yellow line: transmitted signal, blue line: recovered signal, purple line: recovered ; top half: global view, bottom half: enlarged view). TOF-FCTL fiber Fig. 8 Structure of the test platform There are two VME crates (A and B), inside each of which there is a module respectively. UT (ETOF-FCTL) is settled in crate A, while a test signal generating module is in B. The test signal generator simulates the FCS behavior in BESIII trigger system to fan out fast control signals. The generated test pulse width is 4 cycles which is the same as FCS. The delay between two successive test pulses is randomly controlled by a seven-order linear-feedback shift register (LFSR). In order to accelerate the test speed, the maximum delay is 128 s. The test procedure is divided into two phases. In the first phase, reset the Seres on the receiver side over and over for calibration. In the second phase, monitor the recovered fast control signal at UT side with oscilloscope to check if there is uncertainty. This procedure is carried out in laboratory and BESIII environment as well. In Fig. 8, the transmitted test signal (yellow line) is used as the trigger source of oscilloscope, 5 Fig. 10 the upgraded ETOF fast control module runs in BESIII 5 Conclusions To further improve the overall time resolution of BESIII TOF detector system, the ETOF system needs to be upgraded. In the upgrade architecture, there are 4 electronic modules needing to receive fast control signals transmitted from BESIII FCS in trigger system. However, there are only 2 transmitting channels between FCS and TOF system, which makes the transmitting topology differs from the existing one. Now, fast control

6 signals are first directly transmitted to 2 electronic modules in BTOF crates and then relayed to ETOF modules by one BTOF module. The uncertainty in fast control signal recovered from Seres occurs in ETOF just as in TOF before. However, there lying the restriction that BTOF modules should not be modified during ETOF upgrade and the fact that the characterization of ETOF new electronics differs from that of BTOF, the existing uncertainty eliminating method in BTOF module doesn t work in ETOF anymore. ETOF modules need to be calibrated again with as less accelerator machine time as possible. In this paper, a new method to eliminate the uncertainty is proposed. In this method, to guarantee the latency certainty, the transmitted fast control data is re-captured under the control of shifted from the global synchronous, not the recovered from Seres bit stream. The exact shifting phase is determined by a TC module embedded in FPGA, which is designed and implemented with the data processing logic in FPGA. Reference 1 E Cerron-Zeballos, J Choi, Hatzifotiadou, et al. Nucl. Instrum. Methods A, 1999, 434(2): H. F. Chen, C. Li, X. L. Wang, et al. High Energy Physics and Nuclear Physics, 2002, 26(3): (in Chinese). 3 J. Wu, B. Bonner, H. F. Chen, et al. Nucl. Instrum Methods A, 2005, 538: H. H. Fan. C.. Feng, W. J. Sun, IEEE Trans. Nucl. Sci., 2013, 60(5): or/a/fast_cntrl_tim_system/ fcts3_0.ps, retrieved 23rd February Christiansen, J., A. Marchioro, P. Moreira, et al. Receiver ASIC for timing, trigger and control Compared with the existing method, it can find out deterministic position with good margin for fast control signals automatically, which makes it better for the application of ETOF upgrade without affecting any other existing subsystems. Results of long time test in laboratory and in-situ BESIII show that the method is correct and stable. Fast control latency uncertainty for BESIII ETOF upgrade can be eliminated successfully. Acknowledgement We gratefully acknowledge r. Hongliang ai, Zhi Wu, Xiaolu Ji, Jingzhou Zhao, Xiaozhuang Wang and other members of BESIII ETOF upgrade group of IHEP for their earnest support and help during the beam test. distribution in LHC experiments, in Nuclear Science Symposium and Medical Imaging Conference Record, 1995., 1995 IEEE, vol. 1, pp B.G. Taylor, IEEE Trans. Nucl. Sci., 1998, 45(3): K. Wang, Z. A. Liu, W. X. Gong, et al, Fast Control System in BESIII Experiment, in IEEE-NPSS Real Time Conference, 2009, pp X. Z. Liu, S. B. Liu, W. Zheng, et al. Journal of Jinlin University, 38(2):483, 2008(in Chinese) 10 A. Zhen, W. X. Gong, Y. N. Guo, et al, Trigger System of BESIII, in proceedings of the 15th Real- Time Conference retrived 25th February C. F. Ye, L. Zhao, S. B. Liu, et al. Rev. Sci. Instrum, 85(4): ,