ATLAS-UK Level-1 Calorimeter Trigger Meeting

Thursday 17 April 2003 at RAL

Agenda

Click this side                               Click this side
for summaries                                for slides (pdf)
 
Hardware
CMM hardware and firmware status........................Ian
ROD hardware and firmware status......................James
ROD test status.......................................Bruce
Bit and pieces........................................Viraj
CPM hardware and plans..............................Richard
CPM tests............................................Gilles
New TTC decoder, VME tests and VMM..................Weiming
UK tests: Integrating CPM, CMM, ROD.......Norman/Discussion 
 
Online software
Software summary......................................Steve
 
Recent meeting highlights
ROD-crate DAQ........................................Norman
ROS...................................................Bruce
TileCal receivers......................................Eric
 
Discussion: Working group organisation & tasks.........Eric
Front-end rack, patch panel and cable layout
9U ROD design
Latency reduction
Organisation of the project
 
Any other business
Date of next UK meeting

Hardware

Status: We now have three assembled CPMs. One fully working, the second with unusable CP FPGAS due to assembly problems but otherwise operational, used to drive data over backplane to the first CPM. A third needs preparation before it can be JTAG/boundary-scan checked for any assembly errors.

A fourth bare PCB has been delivered, which will be assembled without CP chips. PCBs delivery was delayed due to low yield.

TTC inputs: CPM assumed parallel termination, but TCM outputs were source terminated (as specified) so CPM termination removed. The TCM termination resistors were found to have the wrong values for the specified 100-ohm tracks across the backplane. These were 56 ohms, and should be replaced by 39-ohm resistors. The output biasing resistors also need changing to 390 ohms. If agreed by the collaboration, then an Engineering Change Order will be issued to modify the design of all TCMs.

Note that other modules (PPM, JEM etc.) should also check this!

9U Crate Power Supplies: Three PSU boxes have now been modified to include a mains filter and delay circuit. At Birmingham, powering-on the 9U crate no longer interferes with surrounding equipment, however Bruce mentioned that at RAL one item of equipment is still affected by the power-on of their supplies.

Next version of CPM: Because of timescales, we previously decided to keep any changes to a minimum, so the production CPM will continue to use 8 VirtexE CP FPGAs. The main task will be to re-route all the CP input signals for better quality. Clock distribution will be improved by using more PLLs and attention to layout. The PCB will hold a new TTCdec card with an improved, but different connector. There was a suggestion to mount the TTCrx chip directly on the CPM (dropped following discussion at a subsequent phone conference - EE).

Richard would like the CP FPGAs to use their unconnected Vref inputs so that other signalling standards, such as SSTL2, may be used. A new CPM with this option would still be 100% compatible with the previous version, and even use the same firmware, allowing new versions to be tested in existing crates. The benefits of SSTL2 are lower switching currents giving less noise, and better-defined signal thresholds giving improved timing margins.

CP FPGA clocks: We need one clock to capture the onboard data and at least one more delayed clock to capture data from adjacent CPMs. Considering just the skew due to layout and transmission path differences, the difference between having one clock and having two clocks is small, 1.2ns vs 0.9ns. There is no significant benefit in having three clocks, as the CP chips see onboard data skewed by 0.7ns (due to the two columns of Serialisers). I felt one clock would be adequate, but agreed to design the PCB for two 'backplane data' clocks.

Timescales: RAL Drawing Office will be able to start work on the re-design early July, and estimate layout finished mid-end August. Assembled boards should appear before October.We must complete testing of the present design before August. ROC and HIT outputs have not been connected to real modules, and the CAN microcontroller remains untested. The new modules must be debugged and tested by mid-2004 for the FDR.

Extensive tests have been done on the second CPM. Only four CP chips are connected and so far they have not been tested. A TTC scan of the LVDS input data coming from DSS has been done and shows similar results as CPM no. 1, with all LVDS receivers locking OK. Data have been exchanged between both CPMs, the new CPM successively plugged on right and left hand side of CPM no. 2. Data have been correctly recovered and TTC scan performed without any problems on input links. Some bad channels that were seen when using the loop-back boards seem to have disappeared.

Three more DSSs have been delivered to Birmingham and a CPM could be 3/4 fully populated with LVDS inputs. Both boards are now populated with a new TTCdec card with I2C access. An HDMC I2C part has been implemented and provides a virtual mapping of all TTCrx registers.

The setting of different clocks for the TTC scan could been done per board now, without requiring a TTC broadcast.

Work has been done to synchronize data from playback memory between the two boards. Similar work has to be done for the DSSs but unfortunately one DSS has an old TTCdec card which makes it difficult to have all four working together. Richard has modified the distribution of the Deskew1 clock on board for better jitter, and the operational time window is now more than 1 ns for data coming from the backplane. First BER tests have been done on one channel of one serialiser, with a BER of 10^–13. The pseudo random pattern is only 128 long and the firmware should be changed to a random pattern similar to the one used in the DSS. More BER tests are foreseen, with crosstalk measurement and as a function of the position in the crate. The next step will be to integrate the CPM with a ROD and a CMM at RAL. Although the Readout path has been tested earlier, long-term tests need to be made.

Gilles remarked that the LVDS cables going into the backplane are not well-supported, and as the number of cables in use increases we will need a good mechanical structure to provide strain relief.

New TTC decoder, VME tests and VMM - Weiming Qian (slides)

TTCrx vs TTCrxDec: The definition and classification of jitter was given. The relation of clock jitter and BER was presented. System jitter requirements and many details were discussed. An example of a jitter test from previous experience was given.

Serveral approaches to a better solution than the present TTCdec were presented. We could have all chips on the main module, TTCrx on a daughter-board but PLLs on the main module so that the clocks are cleaned up right near where they are used, or everything on a daughter-board with care taken to feed the clocks to the board cleanly via good connectors and differential lines. This last option would produce a rather large daughter-board. In discussion within the UK meeting the first or second options seemed favoured, but we need to get the views of Heidelberg (Paul) and Mainz (Uli) before we can come to a decision. However, the timescale is short if the new boards are to be used on the PPM and new CPM.

VME-- signal test: Several VME-- signals were tested on the 9U backplane. The results show overshoot about 30% and undershoot about 10%. Futher tests will be needed when the crate is fully populated.

VMM status: VMM modules are fixed. The problem is identified as the +5V fuse, which caused a significant voltage drop of about 150mV. The solution for now is to bypass the fuse with thick wire; for the future lower resistance fuses should be found.

UK tests: Integrating CPM, CMM, ROD - Norman Gee/Discussion (slides)

Norman initiated a discussion on testing. There is now a continuous spectrum of testing, from single modules to growing subsystems (viewed both as subsystems and as tests of a single module's interfaces), rather than the planned sequential approach. However, before FDRs can be passed, modules must pass tests with all nearest-neighbour module interactions working. As far as the CP subsystem is concerned, it is urgent to start CPM–CMM and CPM–ROD tests. These will be started at RAL as soon as possible. Thereafter, work at Birmingham should focus on soak-testing with all real-time and readout paths busy (including CMM), and at RAL on system aspects including ROD output and CMM–CMM interactions. Both institutes will need both modules.

Online software

Software summary - Steve Hillier

Recent work has concentrated on trying to properly integrate the current module testing (particularly the CPM and JEM) into the run control environment. This is already partially done, but needs more work. A recent meeting and various discussions were held at QMUL while Cano was visiting, and just after Jürgen had started work there. Jürgen will be working on the JEM simulation when the recently received code from Stockholm has been tidied up by Steve.

Recent meeting highlights

ROD-crate DAQ - Norman Gee (slides)

Norman gave a brief summary of the ROD-Crate DAQ meeting at CERN on 26 March 2003. For details see slides at http://agenda.cern.ch/age?a03444.

There were various announcements. CERN will support diskless booting of Concurrent CPUs. TTC support has been re-organised after retirement of B. Taylor. M. Joos is the contact for the VME modules. ROD Busy Module drivers and library are available, documentation is in EDMS. It was confirmed that there is no ATLAS-standard ROD Crate TTC distribution scheme.

Ralf Spiwoks could start CTPd integration tests from July 2003. He has a Concurrent CPU. The CTPd requires a CERN 9U crate. He uses a feature of ROD-Crate DAQ in which software pulls data over VME from the CTPd into a Software ROB-in, then transmits it over an S-Link from the single-board CPU. Transmission can also be over Ethernet. The system can sustain 20kHz of 80-byte data to S-Link.

There was also some discussion of calibration, and input is invited from Level-1 on our requirements for the next DIG forum.

ROS - Bruce Barnett (slides)

Bruce presented a status report on issues involving the use of ROS for L1Calo readout, an area in which he acts as contact point for the group. He reported a meeting which took place on 27 March, just after the ROD-crate DAQ meeting, which aimed at a clarification of issues of monitoring and acquisition relating to L1Calo use of the ROS. The discussions were useful, and led to the proposal to use a customized dataOut() in the IOManager to help select events. A later discussion with Markus Joos and Jorgen Petersen led to the suggestion to move to FILAR/HOLA technology for readout. This would require some investment, but would keep us in line with mainstream readout developments, and improve throughput in our system. System reconfiguration is underway, and it is hoped to reinstall ROS software quite soon.

TileCal receivers - Eric Eisenhandler (slides)

Eric gave a brief summary of recent interactions, including a discussion at CERN, with Bill Cleland. A complete first draft of the receiver specification has been sent to Bill for comments, and is also available on the web for everyone else to see and comment on, at http://hepwww.ph.qmul.ac.uk/~efe/ReceiverSpec.pdf. The big unresolved issue is whether transformer coupling is acceptable for the unipolar pulses of the TileCal, where they will introduce a small baseline shift. Bill thinks it is probably ok, but to calculate the likely shift we need more information on the rate and amplitude distribution of pulses. A request for what is known has been sent to the TileCal group. Eric is also trying to contact the Rio group, who built the summing amplifiers, to be absolutely sure we understand the gain of the signals – their web pages are a bit ambiguous, and it is easy to make factor-of-two mistakes when dealing with differential signals.

The UK has ordered the variable-gain amplifier chips (which are going out of production) needed for the receivers, after an unhappy experience with a firm quoting an incorrect low price and then deliberately 'losing' the order. The are also indications that the TileCal group is preparing to go out to tender for the long cables from the calorimeters to USA15 immediately after Easter.

Discussion: Working group organisation and tasks (slides)

The items below were identified at Mainz as tasks for the coming few months. To make progress, working groups should be set up to examine the issues and make recommendations. Although not UK-only, they were discussed here to get UK views and to start naming the people who would work on them. Note that in all cases other volunteers would be more than welcome.

Front-end rack, patch panel and cable layout

This item was triggered partly by realisation that we had not thought about specifying and ordering the short cables from TileCal patch panels to receivers, nor the smaller number of cables from receivers to the downstream patch panel (needed to avoid "octopus" cables). In addition, there had been a discussion at CERN on cable mechanics that was relevant (notes circulated by Paul Hanke), and of course we also need to minimise our latency.

The layout should provide ease of access to patch panels, receivers and Preprocessor, and good cable mechanics for support and strain relief. It should then be possible to specify the lengths of the short cables so that they can be ordered at about the same time as the long TileCal cables. The latency must be kept to a minimum. Hopefully a suggested layout could be produced within a month or two, and then distributed for comments by the community before coming to a conclusion.

This exercise should also be used to trigger completion of the cabling document produced by Steve and Murrough last year.

The suggested people for this were: Murrough, Steve, John, and Paul.

9U ROD design

We must define the specification for the 9U full-specification ROD, leading to a Preliminary Design Review. It should incorporate what we have learned so far, plus the requirements of Preprocessor readout – including compression of FADC raw data. We probably cannot decide yet on processing requirements for calibration and monitoring, so a way around might be to make provision for a processor or DSP daughter-board to be defined later. The suggested timescale: is 2–3 months.

Again the suggested procedure is to set up a small working party, and invite all interested people to brainstorming session(s) before writing a draft specification.

John questioned the need for data compression as he felt that we would not often want to read out a large volume of raw data. Viraj and Ian suggested starting from the specification for the present 6U ROD, replacing 6U with 9U and then seeing what needed to be changed.

The suggested people for this were Viraj, Bruce, Norman and Ian from the UK. Sam and Uli might be asked from outside the UK. Steve and Paul are currently very busy but might be suitable as reviewers (along with others).

Latency reduction

Recent measurements on the CPM and CMM have been worrying. We need to try to reduce latency wherever possible, but not to waste precious effort where there would be little gain. It was suggested that a group of people should take an overall look at the problem. First to summarise all present measurements of latency, and define how to estimate or measure missing information. Then to consider possible areas for improvement, such as rack and cable layout, board layout, choice of FPGA device types, and firmware. The evaluation should try to establish where we could make really significant gains in the latency, in order to understand where the problems are and to make best use of our effort. The timescale again might be the next 2–3 months.

Suggested people were Tony, Murrough, and Gilles from the UK. Uli and Sam might be asked, and for the Preprocessor perhaps Pavel.

Organisation of the project

Although triggered by Tony's request to give up hardware coordination duties, this has broadened since our organisation has not been reviewed since it was set up about six years ago and the nature of the work is now quite different. The hardware coordinator’s tasks need to be re-defined (also see below) and for duties needed short-term we need to provide interim cover. For the contact people at our "outside interfaces" we should evaluate their effectiveness and review the definition of their roles (which contacts are now needed, etc.). For coordinators, we should review the structure now needed for hardware, software, testing, etc.

A small committee of senior people should be set up by Eric, with the aim of presenting proposals to the Management Committee at the Queen Mary meeting in July. The entire community should be asked for frank comments by Eric. Eric should also talk to contact people and coordinators to see how they view their roles and effectiveness. Contact people should be asked to summarise their activities.

We should also make sure that when working groups outside the project would like a representative, it is us and not them who decide on who it is.

Any other business

None.

Next UK meeting

Tentatively set for Monday, 19th May at RAL.

Eric Eisenhandler, 6 May 2003

This page was last updated on 16 December, 2013
Comments and requests for clarification may be sent to
E-mail:
Telephone: