ATLAS-UK Level-1 Calorimeter Trigger Meeting
Thursday 17 October 2002 at Birmingham
Present: Ian Brawn, Eric Eisenhandler (chair), John Garvey, Norman Gee, Stephen
Hillier, Murrough Landon, Gilles Mahout, Tamsin
Moye, Ed Moyse, Viraj Perera, Richard
Staley, Peter Watkins, Alan
Watson + Bruce Barnett (by telephone)
Click this side Click this side
for summaries for slides (pdf)
Cluster Processor Module and related cards..........Richard
Cluster Processor Module and tests...................Gilles
Common Merger Module....................................Ian
CP/JEP ROD and DSS firmware.................Viraj for James
TCM, GIO, CAN, TTCrx.........................Viraj for Adam
Online software status
Offline simulation and physics
Trigger simulator and integration status.................Ed
Tower signals and forward rates........................Alan
TDAQ Steering Group....................................Eric
ATLAS week highlights...................Norman and Murrough
LEC Workshop highlights..............................Gilles
Birmingham joint meeting.........................Discussion
Any other business
Dates of next UK meetings
Cluster Processor Module and related
cards - Richard Staley (slides)
Steady progress made in validating the CPM since last meeting. Clock distribution
working with TTCdec, and there is communication between Serialisers and CP chips,
albeit with poor timing margins.
Very difficult to probe the CPM. A backplane extender is being built to allow
clear access to both sides of a module. All signals, except the 160Mb/s FIO
will be connected. The FIO may be looped-back at the module connector, allowing
the module to test itself. Four PCBs have been ordered, and expected 28th October.
For driving test signals into a CPM, a new LVDS Source card is also being made.This
needed a new connector suitable for using the high-density AMP cables. 10 PCBs
are expected end of October.
We discovered the TTCdec cards contained a Version 2.2 TTCrx chip, and not
the version 3.0 as expected. Version 2.2 does not have a I2C control bus, so
Gilles is adjusting clock phase using TTC Commands. Also, the ID is derived
from a PROM and not the Crate Geographical Address. Version 3.0 never made it
to fabrication, and the latest version of TTCrx is only available in a larger
package. It was decided to design a new TTCdec module as soon as possible. In
the meantime, the original TTCdec cards can still be used for the initial testing
of the CPM.
The orientation of the incoming LVDS cables from the Pre-processor has not
been specified anywhere. The polarisation shrouds on the present backplane have
been fitted either way, but are not very effective. Sam has been informed of
this. A worse problem is that the outer columns of the backplane connector are
full length and interfere with the cable connector body. These should really
be trimmed back to the backplane. The should not be removed as they provide
a ground connection for the modules on the other side. There are a few mechanical
ommisions from the design of the PCB. There is no suitable way of strengthening
the current CPM. Also, heatsinks must be removable to allow possible rework
of the PCB. This will be fixed in the next PCB.
More testing is needed before we should give the go-ahead to make further
CPMs. The LVDS receivers are still untested, as is the connection into the Serialisers.
The design of the 160Mb/s links need verifying. The backplane FIO design needs
verifying, and the termination checked on all signals. Connectivity to the HitSum
FPGAs, ROCs and GLink need checking. At a later stage , before the slice tests,
we need to run BER (Bit Error Rate) tests on all the 80 LVDS input links. This
has to be done in real-time, and is possible if the Serialiser FPGAs are re-programmed
to function as a BERT receiver.
In the summary, Richard repeated that the Crate Extender is essential to the
development and testing of the CPM.
Cluster Processor Module
and tests - Gilles Mahout (slides)
The ROC DAQ/RoI have been tested successfully. A L1A trigger request was generated
on VME command and transfer of data between pipeline and FIFO were performed
correctly, taking into account the value of Nslices to transfer. Data were available
on Glink output but we were not able to check their values. A logic analyser
is expected to be used to perform this task. CP chip algorithm was checked by
sending very simple data, scanning different TTs at different energies. RoIs
were correctly identified...except for some areas. To understand the problem,
the ScanPath F/W has been loaded and data value checked at the input of the
CP chip. It appears then that some channel values were wrong by a clock tick
with other channel values. A small delay of 100 ps on some channels , generated
just on clock edge, make them miss the right clock. They end up available on
the next clock. The solution has been to make a new Serialiser F/W where data
are generated not on clock edge but halfway of the clock tick. Therefore the
CP chip recovered the data correctly by adding extra delays, of the length of
the halfway clock tick, and can cope with late channels with extra delays, but
still staying inside the right clock. Data are all on time now but some data
appear to be corrupted. To investigate the problem and check the robustness
of the calibration process, a TTCdec card has been used into the CPM. As two
independent deskew 40 MHz clocks are driven the Srl and CP chips, tests could
be done by varying the delay between these 2 clocks. Non-corrupted data are
transferred successfully on special setting between this 2 deskew clocks, but
the range is very narrow, around 1.5ns. The next step will be to put ChipScope
in the CP and SRL chip and investigate where and when corrupted data appears.
The CP F/W would need to be changed to enable a reset of the DLLs, as when the
clock changes too dramatically during testing, DLLs lost their lock.
LVDS tests are done in parallel with other tests on the CPM. It is foreseen
to deliver LVDS signals at the input of the CPM by using an available DSS. 4
more DSS will be needed in the future to fully populate the CPM input. 18 LVDS
cables are also needed and a set of 7 LVDS cards need to be produced. The Backplane
will be tested with the help of the Loop I/O option available on the extender
board, and available GIO card will test the real time path to the CMM.
In discussion, Steve pointed out the need to have a software timing calibration
of the timing in order not to keep adding to the latency. Timing and what's
needed to make a second CPM were discussed.
Common Merger Module - Ian Brawn (slides)
Ian reported on the progress of the CMM tests. Testing of the real-time data
path has been completed successfully. Ian described the scope and results of
these tests. Work is currently under way to commission the CPLD that will control
the configuration the Crate and System FPGAs. Progress in this area was described.
The following tasks should be completed before the next iteration of CMMs are
produced: commision the FPGA-loading logic; test the readout logic; test the
TTCrx interface. Design of the next iteration of CMMs needs to start by the
beginning of January if they are to be ready for a slice test by the end of
March '03. Eric expressed the view that design changes between the current and
next CMM iteration should be kept to a minimum, and Ian agreed that changes
should be made only where necessary for bug-free operation in the slice tests.
CP/JEP ROD and DSS firmware
- Viraj Perera for James Edwards (slides)
ROD firmware for CMM, JEM slice readout and JEM RoIs has been written. S-link
firmware with smaller buffers to allow use of ChipScope for diagnostics has
been prepared. For the JEP CMMs, the firmware cannot be written yet due to lack
ROD testing - Bruce Barnett (slides)
A single DSS/(CP/JEP)ROD pair has been use at RAL R1 for some
time as a test bed:
- for L1Calo software development
- for the debugging of existing firmware.
Bruce presented the status of these tests. Continued S-link flow
control problems have required implementation of new S-link status registers
in the ROD and debugging sessions with James and chipscope. In addition, there
are a number of transient power-up conditions which cause occasional problems.
These should be addressed in designs of the final RODs. Proposed solutions to
such problems, along with firmware modifications deemed useful for the final
ROD, need to be tracked. Bruce, in conjunction with Viraj, has suggested an
informal mechanism to help do this: a "Request for Modification" form.
Broader understanding of the firmware by all would allow faster
progress in the firmware debug cycle. Firmware tutorial sessions, and read-only
access to the firmware designs through the hdl_designer would be helpful in
this direction. Addressing the former, James has very recently given a tutorial
session to familiarise Norman, Bruce and Eric with the ROD design.
The testing software has progressed from a stand alone "Looper"
code, to a code integrated into the DAQ –1 environment (Work of L1Calo
s/w group.) In this form, the s/w may be used for Module acceptance tests described
at previous meetings.
For the future, there are a number of firmware issues to be resolved
(details of status-word zero-suppression, handling of error-bits). In addition,
new firmware variants (JEM, CMM, etc.) await testing, and new DSS functionality
(wrap-around) needs to be incorporated into the s/w.
TCM, GIO, CAN, TTCrx - Viraj Perera for Adam
TCM - The VME display has been fixed, and the memory extended
for CAN. Three modules are in use, three others need to be updated.
GIO - The design has been modified, and is ready to go to the
Drawing Office to make more. Should four be ordered? Tony felt that might not
be enough; it will be discussed.
CAN - The TCM can now request temperature and voltage readings
from the CMM. On the CMM, temperatures are obtained via SMbus and voltages are
digitised in the Fujitsu's ADC. The CMM sends an alarm if it is too hot. Software
for the CPM has been started. The question of how to interface the TCM to the
outside world was raised: ELMB only, ELMB and Fujitsu, Fujitsu only? Each has
pros and cons.
TTCrx - Needs testing. Design of the card needs updating for
the new TTCrx chips.
Online software status
Online software status - Murrough Landon (slides)
Murrough presented the status of our Online software developments. He showed
a table showing extent of progress or remaining work in various areas for the
different subsystems, though the figures in the table were fairly arbitrary.
The general infrastructure and specific code for the CPROD test system is
now in a state where we can make a frozen release for use in testing ROD/DSS
modules by non-software experts. This is a satisfying milestone to have passed.
A lot of work in many areas eg HDMC, module services for CPM, simulation,
run control, IGUI etc has been done. However much remains to be done, even for
the CP subsystem tests. The extensive but standalone work on the JEM needs to
be integrated with our framework, and there is still more to do in the preprocessor
Module services - Bruce
Bruce outlined a number of software improvements in the area of the "moduleServices"
package. The HDMC PartManager:: readDaqPartsList()) method was introduced to
allow the reading of hdmc.parts files containing composite structures in an
effective way, both from program code and the HDMC Gui. A bug in PartManager::remove_part(recursive)
was isolated, so that the operation of readDaqPartsList(), is now stable.
Bruce reiterated the basic function of the DaqInterface methods load() and
config(), and indicated the need for a number of new methods to be added to
that interface (eg: verify() and identify()). Progress on CMM (Norman) and CPM
(Gilles) variants of moduleServices continues.
A number of needs, in particular complete modulesService support for the DSS
LVDS daughter card (urgent) and DSS-CTP emulator (using the GIO card) were outlined.
Bruce has plans to address these soon.
Simulation - Steve Hillier
After the major upheavals of the summer, recent work has consisted of tidying
up a few details and minor improvements for the notional CP ROD test release.
Two areas that needed changing were the method of obtaining the test descriptor
from the database, and sending of MRS messages when there is a problem with
the test vector generation. These have been done and tested. On the ROD simulation
itself, careful work with new test-vectors has identified problems with Bill's
original ROD output files, which have been corrected in the new simulation.
However, one or two issues are still causing the simulation to disagree with
the hardware in an area where the documentation didn't fully specify the data
Offline simulation and physics
and integration status - Ed Moyse (slides)
Ed reported that Alan has found a few issues with the Em/Tau trigger simulation,
most of which are fixed. The JetTrigger is finished, and the Energy trigger
is nearly so. Both still need debugging.
Ed further explained some of the problems he's had with the persistant storage
of TrigT1Calo objects, and the ElementLink class that appears to the best solution,
despite the lack of documentation. Ed expects to have finished coding by mid-November.
Tower signals and forward rates
- Alan Watson (slides)
Alan presented updates on a few barely-related simulation matters:
The new Linux farm has arrived at Birmingham, and is being commissioned.
Validation test with Ed's TrigT1Calo uncovered a few minor bugs, which have
now been fixed. Comparisons of results with Atrig show good consistency, though
not all distributions are identical (and probably shouldn't be expected to be).
New Data Challenge 1 datasets show effect of increased material – calibration
adjustments will be needed.
Requirements for trigger tower simulation have been discussed with the tile
and LAr groups. PPr simulation will also be needed to accompany this. All of
this should be available by the end of the year.
Updated studies of forward jet rates revealed an error in earlier estimates.
Stand-alone forward jet triggers are probably not viable, but in combination
with other signatures the rates and thresholds look useful. Physics would prefer
a more flexible definition of "forward" to "including FCAL"
– the feasibility and consequences of this will need to be studied.
TDAQ Steering Group - Eric Eisenhandler (slides)
Eric very briefly summarised what had been discussed at two recent meetings
of the Trigger/DAQ Steering Group. See his slides for the agendas, and go to
the CERN agenda system to find the talks.
ATLAS week highlights - Norman Gee (slides)
and Murrough Landon
Norman summarised various points from the TDAQ at the ATLAS week 7-10 October
2002. Following TDAQ reorganisation, Data Collection is overseen by David Francis.
It includes the ROS, Parts of ROD-Crate DAQ, and the RoIBuilder plus the former
Data Collection. Ralf Spiwoks is interim coordinator of the ROD-crate DAQ, and
Bob Blair is responsible for the RoIBuilder.
A revised RoIBuilder design features two 9U input boards with six full-spec
S-Link inputs from (Level-1) RODs, and up to four RoIBuilder modules each feeding
up to four Level-2 Supervisors. Events are duplicated by the input boards and
fed to all RoIBuilder boards over Gigabit Ethernet or LVDS. The RoIBuilder boards
exchange a token over VMEbus lines to agree which one of them will process each
event, and other RoIBuilders ignore the event. In general, there is much more
attention to details of flow control, error handling, performance monitoring,
In the TDAQ plenary, an increased deferral of –13MSf –3.9 –4
was presented. This has a severe effect on trigger rates. The DAQ/HLT TDR first
draft is planned for January 2003 with submission in June 2003. Marc Dobson
will be technical editor.
At the DIG, the proposal to insert run numbers into ROD event headers was
presented - this now seems to be accepted. Beam tests by Tilecal, Muon MDT,
and Pixels have taken place with the central DAQ software, including a common
run. Atlas DAQ will move to a system where all software emerges in releases,
including utilities contributed by detector groups. Pascal Perrodo presented
the LAr calibration strategy, which runs independent from the main DAQ system.
At the Electronics meeting, C Parkman showed the rack allocation database
and gave details about crates. Heat deflectors will be riveted into racks, so
the crate layout cannot easily be changed. Crates are being ordered in batches
just once per year. Orders are being collected in October/November each year
for delivery in the first half of the following year, and we need to decide
very soon for this year. P Farthout described a proposed Local Trigger Processor
now being specified, which will generate detector test signals and allow combined
runs between level-1 and calorimeters without use of the CTP. We need to check
the specification carefully when it arrives.
The TDIB included a discussion of the options for deferring cost, by multiplexing
S-Links emergying from RODs, or by increasing the number of Robins per ROS (i.e.
decreasing the number of ROS), or by multiplexing the ROS outputs.
Outside the formal sessions, there were several useful meetings. Agreement
was reached on an outline calibration scheme with the Tilecal and LAr groups.
Planning is needed for RoIBuilder and CTPD patch panel tests.
Norman also described a first measurement of cpu time needed for calibrating
energy and pulse timing. The main computation needs to be done on ROD daughter
Murrough had various discussions with members of the Online group. In particular
some of our developments in the database area will be adopted as part of the
We have been invited to give a talk about our use of the Online software at
the next Online meeting (13 November, the day before the ROD workshop).
LEC Workshop highlights - Gilles Mahout (slides)
Gilles showed a selection of highlights from the recent LEC Workshop in Colmar.
See his slides.
Birmingham joint meeting - Eric Eisenhandler
Eric discussed tentative arrangements for the upcoming Joint Meeting at Birmingham.
There was no other business.
The next UK trigger meeting was tentatively set for Wednesday, 11th December
at RAL, but since then a conflict has emerged and a new date is needed.
6 November 2002