ATLAS-UK Level-1 Calorimeter
Trigger Meeting
Tuesday 5th June 2001
Rutherford Appleton Laboratory
Present: Bruce Barnett, Ian Brawn,
James Edwards, Eric Eisenhandler (chair), John Garvey, Norman Gee, Tony
Gillman, Bob Hatley, Steve Hillier, Murrough Landon, Gilles Mahout (minutes),
Ed Moyse, David Mills, Viraj Perera
AGENDA
Generic Test Module - Viraj
CP "chip" - James
... and does it 'lose' bottom or top row? - Steve (slides)
Cluster Processor Module - Richard
... readout controller design and test plans - Gilles (ROC)
and (testplan)
CP/JEP ROD prototype - Viraj
Common Merger Module specification - Norman
... and design - Ian (slides)
Parallel LVDS transceiver test card - Viraj (slides)
Timing Control Module & adapter, CPU adapter - Bob
Crates and power supplies for slice test - Bob
JEM and backplane status summary - Tony (slides)
Short-term schedule - Tony (comments on schedule)
and (charts)
CANbus: a brief introduction, and Fujitsu status
- Dave (slides)
ROD tests - Bruce (slides)
Thoughts on simulation and test vectors - Steve (slides)
Concurrent CPU purchase situation - Norman
TileCal trigger-tower situation - Eric
Preparation for Mainz joint meeting and T/DAQ week - Eric
Any other business
Dates of next meetings
MINUTES
Hardware status
Generic Test Module - Viraj
Two new modules are going to be made following problems encountered with
previous ones. PCBs are already there and components are going to be delivered
second week of June. The board is scheduled to be available at the end of
June. The assembly is going to be done by the same company as with the previous
boards. Concerning the two old boards, one of them has been lost by the
re-working company, STI. Investigations are being made to see if we can
claim some money back.
CP "chip" - James
Version 1.0 of the CP chip firmware exists. The design meets the specification.
Although James mentioned that the test vectors were wrong to simulate correctly
the CP chip design, Steve pointed out that the format does match the CPM
specification which is unfortunately different to the present design of
the chip. He can easily change the test vectors to the format of the present
CP chip design but a decision has been made to use the CPM specification
providing Ian re-does the labelling of the serialiser. In this case, James
does not need to change the complex algorithm of the CP chip. See the following
item on the loss of the bottom or top row.
... and does it 'lose' bottom or top row? - Steve
(slides)
Steve explained how a confusion has happened between the CPM specification
and the present CP chip design. Eight elementary trigger towers (2x4 TT)
are processed by a CP chip. The algorithm involves the use of 5x7 TT in
order to surround these 8 towers. The problem occurs due to the BC-mux delivering
pairs of TTs always multiplexed in phi. The CP chip has to deal with 6x7
in input and so needs to eliminate one extra row in phi. In the CPM specification
the bottom row is eliminated, but in the CP chip design the top row has
been discarded. Both sets work if they are applied correctly.
The CPM specification choice is justified by the fact that the eight TTs
to be processed match the eight TTs coming from its serialisers. The input
of the serialisers also match the output of the PPM which delivers the same
8 TT. Therefore, in case of a problem in one TT, it is easier to track down
the faulty tower to its respective cable. In addition, the phi boundaries
really are at multiples of 90 degrees.
In the present CP chip design, the 8 cells processed are in two different
cables, which will add confusion. Furthermore, Steve pointed out that there
is already some confusion between the PPM and the CPM since signals coming
from the top of the PPM will be connected to the bottom of the CPM. Murrough
added that a lot of documents already exist showing the CPM scheme against
only one with the present CP chip design.
James said that to redo the algorithm is very delicate, but another solution
is to change the reference cell delivered by the serialiser. Ian said this
was possible by introducing a new awkward labelling scheme from the serialiser
with negative values. The job could be done in less than one day. It was
therefore agreed to do this, and stay with the CPM scheme from now on.
Cluster Processor Module - Richard
Richard was not present but Tony gave a short summary of the CPM. The board
has reached 45 sheets of schematic now. It's nearly finished as the layout
now requires only the schematic of the logical devices to be used for the
Readout Controller (ROC) and the hit merger. Gilles has finished the design
of these two blocks and the functionality test shows that they are working
correctly. The hit block has been targeted in Virtex-E XCV-50 device.
... readout controller design and test plans
- Gilles (ROC) and (testplan)
Concerning the ROC, the functionality of this block has been described.
So far only the output of the data path to ROD has been designed. It consists
of slices of 20-bit data coming from the 20 serialisers, each data having
a length of 80 bits. Then comes the 48 bits of the hit merger data, together
with the 12 bits of the BCN. These two are merged to form the 20-bit slice
output, where the first 16 first bits consist of the hit data and the other
four come from the BCN. Its functionality test works and it has been targeted
successfully in Xilinx XCV-100. The number of pins left is very low, 98%
occupancy, and the design needs to be modified as some unnecessary signals
have been taken into account. It is hoped to decrease to 85-90%. The timing
simulation is not satisfying and it needs some investigations.
Regarding the RoI ROC, it has been agreed to use a second device to give
more flexibility to the board for the future. Viraj asked to use Virtex-E
instead of Virtex.
An update on the test plans was also presented. For the boundary scan,
JTAG connectors have been chosen, for the FPGAs Xlinx Xchecker, and for
the CPLDs Altera Byteblaster. The CPLDs are not reconfigurable via VME and
need the help of an external PC. For the tests of the board, it has been
thought to build a local crate to test the CPM if there is nothing available
when the board is ready. It will be built to hold at least 2 CPMs, a 6U
CPU unit and a TCM.
With only one CPM in the crate, VME access, TTCrx and micro-controller
access will be tested. Deeper tests will involve testing the CP chips with
the help of test-vector playback from the DP RAM of the serialiser. The
memory provides a 40-bit test pattern sent to the inputs of the CP chips.
By selecting the Real Path configuration for the CP chip algorithm, 432
bits are output from the 108 inputs of the CP chip. 54 pins are dedicated
to the EM part and the other half to the hadronic, therefore we can use
the same test vectors to probe both sides of the connectivity. This 40-bit
pattern could consist of 10 packets of 4 bits each, each packet dispatched
along the 10 serial lines driving 8 trigger towers. These packets could
also be flagged in order to know if they are coming from 4 TT on the right
or 4 TT on the left, helpful to track them back once processed through the
fanout. We assume here that TTs are looped back via the backplane to the
input of the CP chip under test.
BC-mux could also be studied by sending test vectors scanning the 9 cases
of BC-mux listed in the TDR. With the help of a second CPM board, part of
the CP algorithm could be tested. Similar tests with the serialiser chips
requires the help of LVDS signals. The serialiser connectivity could be
tested by switching off the BC-mux process of the serialiser chip, retrieving
and checking the data from the DP memory.
The tests of the ROC, hit merger, etc. could be done with probes on test-points.
All those tests require a lot of software to be written, from a sample R/W
register to a complete program to test the CP algorithm.
CP/JEP ROD prototype - Viraj
Viraj explained they need to go through the issues encountered at the CERN
tests. The remaining two boards need to be assembled, and they need to manufacture
four new boards.
There are other variants of the firmware to be developed as well, since
only the CP slice readout and the CP RoI readout firmware is available at
the present moment.
Common Merger Module specification - Norman
Version 1.0 of the CMM specification is now available on the web. The hit
signal I/O will use 68-pin SCSI connectors, which are metal housing, more
robust than IDC ribbons, and commercially available. But the CTP people
(Georges Schuler) have commented that this solution is not reliable. Contact
has been made with him to find a better solution and we are waiting for
an answer.
Tony pointed out that we don't need a halogen free solution for now, but
it will be crucial later. The schematic cannot be finalized yet as the choice
of the connector is one of the last items to be decided.
... and design - Ian (slides)
The first version of the CMM schematic is available on the web at http://www.te.rl.ac.uk/esdg/atlas-ft/specs/cmm/.
Some buffers need to be changed to cope with 5V tolerance issues relating
to Virtex-E devices. The problem of choice of connector is also delaying
the final version. Hopefully, an answer is expected soon and a new version
could be issued at the end of the week. Schematic reviewers will be required
and Sam and Uli have been thought of as obvious people to deal with the
CMM via the backplane and the JEM. But Sam is very busy and another person
needs to be found. Meanwhile, the schematic has already been internally
reviewed.
Parallel LVDS transceiver test card - Viraj (slides)
Viraj reported on the status of the LVDS parallel-data CMC I/O board for
the DSS module, to be used with the CMM. It is made of 33 differential I/O
pairs, or 66 single-ended. The card can be either a transmitter or receiver,
the flexibility coming from its implementation in Xilinx Virtex FPGA (XCV300E-BGA432).
Tony asked about the timing delay, which is 6 ns. It can also generate
other I/O standards (LVCMOS,...). Only one PCB design is necessary, the
core of the configuration being in the FPGA device. Fixed test-patterns
could be generated either from the logical device on board the CMC card
or from the DSS. Delays could be generated to test synchronization, and
a PCI interface has been implemented.
The connection to the DSS is via three CMC connectors, and a forth connector
is dedicated to the GTM. JTAG ports are used to program EEPROMs and access
FPGA. The assembly of the board depends on its use: LVDS receiver, LVDS
transmitter, single-ended CMOS,etc. Once a CMC board is set up, it is unlikely
that we'll need to change it very soon, so a pluggable version has not been
considered. The connectors foreseen are the 68-way SCSI-3 connector, despite
some concerns from CERN (see above). This latched connector fits very nicely
to the board, and it has round cables with screen. ISIS uses this kind of
cables, with length 10 m, without problems. Some concerns have been expressed
concerning the amount of I/O between the CMM and the CTP, and contact should
be made with Ralf Spiwoks.
Timing Control Module & adapter, CPU adapter
- Bob
Bob reported that the three types of PCBs are all manufactured and awaiting
assembly. Most of the components are available except for LEDs, one IC type
(but an alternative exists), and power connector sockets.
Three off main TCM cards, all display PCBs, and all adapter-link cards
are going to be sent for assembly at the end of the next week, with an approximate
one-week turn around. An estimated four weeks per module will be required
for the mechanical assembly and functional testing. A pseudo-backplane will
be necessary. Documentation needs to be updated. Concerning the female connectors,
they have been ordered and we are committed to a 22-week delivery. Uli will
use the same ones for the JEM.
The design of VME mount module (also called Personality Card or CPU adapter)
has been agreed and submitted to drawing office for layout. The start of
the layout is subject to the availability of drawing office effort, in competition
with other ATLAS LVL1 trigger jobs (CMM, CPM and CMC card). Some priority
will be needed and the next effort available will be mid July. The mechanical
design is 90% completed and the metalwork for the VME processor cage is
in the workshop. Four modules are going to be built, one for each crate.
As an additional comment, Bob mentioned that 7 TTC decoder cards have been
assembled. They require testing now. It is based on the old PCB design for
holding old available ASICs from CERN.
Crates and power supplies for slice test - Bob
All the components for four sets of crates are available (PSU blocks, chassis,etc...).
Wiring layout and 50% of the mechanical layout have been done. Assembly
of a prototype is to begin this or next week, for a two-week test. The next
three would be assembled at Birmingham. There is still question whether
to supply the fan tray from the DC supply or not.
JEM and backplane status summary - Tony (slides)
Tony summarized the status of the non-UK hardware. A call for a tender
for the backplane was put out at the beginning of May but quickly delayed
due to some bureaucratic hurdles. The closing date was Monday 4 June. The
backplanes will be available at the earliest by September 2001.
Concerning the JEM status, only one company made a bid for PCB manufacture.
The first delivered board was not a success, and the second one has a lot
of defects. There were some bugs in the layout not picked up by Cadence.
The first and only JEM board was available in May, but the Spartan II BGA
device package was not put on correctly (rotated by 180 degrees). The rework
of the board was successful and preliminary tests are on their way.
It may be necessary to iterate the board design to fix the known bugs before
the Slice Test. This will be discussed at the Mainz meeting. Meanwhile,
Uli suggested participating to the regular UK hardware progress meetings
at RAL, via phone or video conference.
Heidelberg people are still in period of transition between students who
have just left and new people arriving. The MCM is coming soon but the ASIC
has not yet been sent out.
Tony commented on the big slippage over the last three months. The desire
to have a unified design (modules and components) has led to an improved
system, but has taken much longer to get agreed specifications. The design
of prototypes to module-0 specifications was very challenging and has required
extra time. Furthermore, we have problems giving realistic times, with a
diplomatic tendency to underestimate it. From the optimistic point of view,
once the design is complete and layout, manufacture and assembly processes
have started, we have a better-understood and hence more accurate time scale.
But the duration of the test programme is very unclear. From the experience
with the ROD, it sounds like three months of testing for pre-Heidelberg
sub-system tests is very optimistic. There is general agreement that we
will use boards in the slice test only if they seem to be bug-free. Tony
expects the beginning of the slice test not before March 2002. The new milestones
are available on the web.
Testing
CANbus: a brief introduction, and Fujitsu status
- Dave (slides)
Dave gave an introduction to CANbus with application to the Fujitsu micro-controller.
CAN stands for Controller Area Network, a multi-master serial network. The
protocol is CANOpen, a priority-based network, i.e. a high priority message
will not be held up by a low priority message.
Two standards define the CAN frames, the difference mainly being the length
of the identifiers (ID): 11 or 29 bits. Both standards allow for a maximum
of 8 bytes of data to be transmitted per message. Each node of the CAN network
has a unique ID (11 or 29 bits) which drives the priority of the message.
Low values mean highest priority messages. CANopen claims the first 4 bits
of the ID to assign message object type, leaving 7 bits to identify the
modules in a crate with. BUS arbitration is done by using bit-by-bit arbitration.
Each node sends the bits of its message ID and monitors the bus level: as
long as the ID bits are identical, each node keeps transmitting its message.
As soon the bit sent is different to the bit received, the node switches
to receiving. For any nodes, an acknowledge bit is sent back regardless
of whether the node is configured to accept the message or not. If the acknowledge
fails, the message is sent again and an internal error counter is incremented.
Dave presented a proposed architecture for the TCM and its interactions
with the DCS. After a set of initialization and reset, four steps are processed:
ID check, check of all modules ready, check if all modules answer, and behave
correctly. A fifth step consists of checking the self-analysis report of
a module. If one of the previous tests fail, a high alert priority message
is sent or details of the problem are investigated and a medium alert message
could be sent.
Dave also presented the architecture of the code of one module talking
to its TCM. Values of voltages and temperatures are stored in the RAM of
the micro-controller, read on request or overwritten with fresh data. The
check of the values is done by comparing them with stored values within
definite limits, the results sending a high alert message or not to the
TCM. Murrough pointed out that those values are stored in the database but
are not from SCADA.
Concerning the Fujitsu micro-controller development, most of the code is
now tidy and readable. The Fujitsu RAM available is 128 kB and a reasonable
list of probes should be defined, with their tolerance. Dave is also documenting
the code and the API. The message data format and the network operation
need to be specified. The testing of the code will require a running demo
system
Software status
ROD tests - Bruce (slides)
Bruce has updated his recent work following the ROD CERN test. During the
tests, there were network problems, bus errors, system hang-ups and memory
violation, inconsistancy of memory match and segmentation faults. The problem
was due to the CPU and the use of old drivers. With a new CPU and new drivers
(1 GHz, Linux 7.0 but old kernel 2.2.16, drivers for INTEL D815EEA, Atlun02,
Atlun03 names), the system is runs far better and will be used for the slice
test.
Bruce has cleaned the ROD test software. He now uses the VME access classes
from HDMC (ModuleMemory32, ModuleRegister32...) instead of the stand-alone
code used at CERN. Test-vectors component, his own and the ones from Bill's
DSS loader, will be updated to use HDMC classes. In order to include the
code in the Run-Control type environment, he has rewritten it in C++.
Concerning the actual testing, nothing has been really done. He has talked
with Viraj and James to define the next stage of tests. Following the S-link
problem at CERN, Norman asked if a diagnostic kit is still necessary? Should
we order a handy logic analyzer from HP? Two S-link destination cards have
been ordered. Tests could be done with the help of hold'in Source/Sink/PCI
interface and optical bits.
Thoughts on simulation and test vectors - Steve
(slides)
Steve presented some software strategy thoughts on hardware simulation.
The motivations are to help in the online monitoring and the test vector
generation. The online monitoring requires data from the same event in several
places, and simulated system response that for input data. It also check
for differences. The test vector generation is a complex system due to the
huge number of components. It is difficult to maintain ad-hoc test vectors
for each part of the system and to track the exact response everywhere.
Vector generation and storage need to be rationalized.
Steve has made a first attempt, learning from his previous work when he
generated test vectors for the CP chips. He has created simple set of C++
classes, added BC-mux to already-existing data, data I/O formatting, and
used simple pre-defined file formats for a single chip. This work has been
very successful as he spotted some inconsistency between the CP chip's specification
and the actual CP algorithm of the chip. Furthermore, the data i/o class
was re-usable As Ed was able to use them for his own offline simulation.
Troubles come when you try to scale to a larger system it's difficult to
fit algorithms into architecture. The current situation is a very simplified
VHDL framework. Two types of object are created: a DataPort (cf VHDL port)
as an interface between processes, and a ProcessElement (VHDL entities)
as processes and algorithms. From these objects, you can build a hierarchy
of elements. So far, there is no detailed timing simulation except one step
meaning one tick. File i/o is provided by specialized processes, and the
architecture is currently configured by one procedure which creates modules
and their connections. In the case of the CP block, the input is constituted
by the ChipData containing 5x7 cells, the outputs are the HITs and ROIs,
all connected together by the CP algorithms. Individual blocks are built
into a container object where other blocks appear, such as "RoI-Builder",
"EM demux",etc. and are connected together. Each block hides its
complexity internally, and externally it appears as a non-composite process.
The status of the work is an almost complete CPM, which will be required
to be connected to other CPMs via a backplane. The idea is to simulate the
slice test with at least four CPMs. Norman is interested to have code to
do the same for the CMM. As it is still at the level of prototype, Steve
said he will clean his code a bit and add more comments. For the tests,
it will consist of original test vectors fed into this. The difficulties
come to know if the results are correct, and how to handle the errors. Offline
simulation should be able to help. Steve added that he is currently working
on integrating Bill's test vector generation techniques.
The next steps will consist to apply what it has been done to the CPM to
the ROD, and test against Bill's work. More complex test vector generators
are needed for detailed study of components. More work on I/O data formats
and its integration into the ATLAS standard needs to be done.
There was discussion on the unidirectionality of the system. It might be
very useful to work backwards, but it is not obvious as several inputs could
give the same output. ML commented about the fact thatATLAS standard documentation
is needed and there can be some overlaps with the existing toolkit developed
by LVL2 doing modeling.
John Garvey pointed out that test vectors for the CP should be similar
to test vectors for the JEM. It was also said that the issue of thresholds
has not been addressed, and we should have test vectors ready before the
first modules arrive.
Concurrent CPU purchase situation - Norman
Norman has ordered three new Concurrent CPU-1s, which are different from
the one Murrough's. Both boards use the Tundra Universe chip to talk to
the VME, but the new one is modified to handle bus errors and other options
(network boot, USB access,...). SCSI will be replaced by EIDE bus to save
space for the bigger chip. Murrough's CPU won't be plug-compatible with
the new board. Mainz also has the old version of the CPU.
Other items
TileCal trigger-tower situation - Eric
Eric and Alan attended a video conference on 11 May 2001 held by the new
leader of the TileCal community, Rupert Leitner. The aim was to agree on
the number and grouping of TileCal trigger towers for each SuperDrawer (phi
slice). We are still waiting for them to formally agree with the summary
of the meeting drafted by Eric [It has now been approved - EE]. The choice
is 9 towers from the barrel and 6
from the extended barrel, i.e arranged as in the proposal discussed at
the summing amplifier FDR in April 1999. The small adverse effect that this
might have on the tau trigger in the bins of eta from 0.8 to 1.0 was considered
to be acceptable in order to facilitate having numbers of TileCal towers
that match to the LAr calo towers. It avoids the trigger to have to do any
additional signal summing.
However, to keep the option of a future improvement open, every effort
will be made to use long cables with at least 10 twisted pairs for the barrel
and 7 twisted pairs for the extended barrel, possibly at an extra cost.
Preparation for Mainz joint meeting and T/DAQ
week - Eric
Eric pointed out that the registration for the Mainz meeting is over. Nobody
from Stockholm will be there as they are busy moving building.
The software brainstorming session of Wednesday afternoon has been canceled.
It has been proposed to hold the next joint meeting at RAL instead of Heidelberg.
A date has to be chosen, probably end of October.
Concerning the TDAQ week, no agenda is available yet. The slot of the Institution
Board is the only one fixed. Only two people from the entire level-1 trigger
were at Brookhaven.
Any other business
Norman pointed out there is a new WG leader of the ROD: R.Mclaren, and
a meeting will need to be arranged between relevant people.
We discusssed the slides of Murrough to be shown on the following day at
a video conference on level-1 online software between CERN, KEK, Kyoto,
Weizmann and RAL.
Dates of next meetings
The next UK meeting will be on Tuesday, 17 July in Birmingham.
| 
The ATLAS Level-1 Calorimeter Trigger