ATLAS-UK Level-1 Calorimeter Trigger Meeting
Thursday 19th April 2001 - RALPresent: Panagiotis Apostologlou, Ian Brawn, Eric Eisenhandler (chair), John Garvey, Norman Gee, Tony Gillman, Bob Hatley, Stephen Hillier (minutes), Murrough Landon, Gilles Mahout, David Mills, Ed Moyse, Viraj Perera, Richard Staley.
Agenda
Click for Click for
summary transparencies
*=ps #=pdf %=gif
Minutes of previous meeting
Hardware status summaries
CP "chip"...............................................Ian 10'
Generic Test Module...................................Viraj 10'
Cluster Processor Module............................Richard 15' #
CP/JEP ROD prototype..................................Viraj 10'
Common Merger Module specification...................Norman 10' #
Common Merger Module design.............................Ian 10'
Timing Control Module and adapters......................Bob 10' #
Crates and power supplies for slice test................Bob 10'
Summary PPM, PPr-ROD and MCM reviews...................Tony 15' #
Short-term schedule and reviews update.................Tony 10'
Testing and Integration
Work on Fujitsu CANbus solution........................Dave 10' #
Integration of CP/JEP ROD with RoIB and ROS..........Norman 15' #
Software status
Online s/w status & comments on T/DAQ week.........Murrough 15' #
Work on new trigger simulation software..................Ed 10' #
Other items
TileCal trigger-tower situation........................Eric 5' #
How many G-Links should we order? ==> ROD numbers
and modularity revisited......................Norman 10' #
LEB and IEEE submissions...............................Eric 5'
Any other business
Dates of next meetings
Hardware status summaries
James and Ian had built the VHDL test-bench for the CP chip code.
James had recently succeeded in getting the test-bench running after
sorting out a synchronisation logic bug, so he should get some results
soon. It was clarified that all these tests were pure software simulation.
Viraj summarized the situation as that there now was no GTM. Of the two
modules sent for re-working, one was usable with some bits missing, and
this had been given to the CMS group. The other was still unusable after
a second re-working. There had been no feedback from the company on how
the re-work had gone, or if any X-ray assessment of the failure modes had taken
place.
Viraj now feels that the current boards will never work, but at
least some lessons could be learned from the experience - in particular that
re-working even a simple board may prove difficult. A new one will
be built, and we will have to pay for parts up front and hope to claim
back some of the costs. Some study of the pads on the failed boards could
prove useful but perhaps the biggest concern is whether to stick with the
current assembling company or try a new one. One of the reasons for initially
choosing this one was that many others do not have the facilities for re-working
modules. For the CPM we will need to build a good relationship with a reliable
company, and to do this may require the production of a trial board first.
The current status of the schematics and PLD coding for the CPM were presented.
The schematics are about 90% complete with the major incomplete areas being
the Read-Out Controller, Hit logic and backplane connectors. Most of the coding
was complete but some still had to be assigned to a device.
Richard said that the timescales were still vague, but he still hopes to get a
module for testing at Birmingham by September. There would be a schematic review
in early May, meaning it should be complete by mid-May. It was commented that
allowing a reasonable time for testing, it seemed unlikely that the CPM would
go to Heidelberg before 2002.
A decision needs to be made on assembling the two remaining PCBs and ordering
four more PCBs for the slice test. The current problems as found at the
integration tests should be reviewed before going ahead, but it was felt that
there would be no major obstacle. The main concern was that all the problems
should be identified as firmware bugs, and that no changes to the motherboard
are needed. The two PCBs could be ready in two weeks,
and the others would probably take about two months to produce.
Tony was interested in how long the ROD testing had taken as an indication of
future work. About three full months were needed and so far only a couple of
the FPGA programs had been tested, although the others should be easier now, but
on the other hand, the work was not yet complete. Murrough asked about numbers
of DSS modules - Viraj said that 9 existed, 3 having S-Link cards compared to
8 needed for tests. The modules are essentially ready to go, but more firmware
may need to be written.
Norman presented the recent updates to the CMM specification. There were four
areas that needed finalizing:
- The Jet ET hits read out an RoI - Norman presented the new proposed data format
including this information.
- The backplane pinout - this was brought into line with Sam's backplane
specification 0.9b2 which provides 3x27 available pairs for the 3x25 needed
for the CMM system. There is a slight oddity with one pair which is split
over two connectors, and Norman has suggested to Sam that he try to change
this if possible.
- The backplane/cable adapter board - a simple layout was shown.
- The TTC registers - these are proposed to be the same as the CPM.
The new version of the specification includes all this work, and it just
requires Sam's confirmation of the backplane change before it can be signed
off. Eric commented that we currently have a lot of specifications that are
not quite signed off.
The original idea of fitting all the logic into one FPGA could not be implemented,
so now the logic is spread over two FPGAs, and consequently these must be devices
with large pin-counts. This meant that Panagiotis had to spend a lot of time
entering the pinout, and Ian has been dividing the firmware into the devices.
There should be spare capacity now with the split firmware.
The work was mostly done, and a review was planned for the
end of April.
It was suggested that the review should perhaps be a phone conference to include
input from the Jet/Energy designers. Suggested reviewers for the schematics
were Uli and Sam from the Jet/Energy side and Richard from the CPM side. There
was a slot booked at the drawing office imminently, and Ian hoped that the design
could be ready in time to not miss this slot. It was commented that nowhere
in any specifications were the exact meanings of 25 signals defined. In principle
this does not matter as they are FPGA to FPGA connections, but it was decided
that they should be labelled bits 0-24 with 24 being the parity bit.
The TCM was now complete up to the stage of being fully routed, and the order
was to be sent within a week after some final checks. Components were being
ordered - five of each in the case of cheap components, and four for expensive
in order to build four final working modules. The layout for the adapter cards
was complete, and the routing was not expected to take long. There had been
nothing new from Paul on the pre-processor version.
The schematics for the processor mounting module were almost complete and
should go to the drawing office soon. Bob showed a picture of the mounting mechanics.
The mechanical design was still in progress. Norman asked for confirmation that
the adapter won't clash with a J0 connector.
News on TTCrx ASICs was that seven prototype ASICs had been found which could
be used with the original TTC decoder card design. This means that 10 decoder
cards could be made which should be enough to keep our systems running until
the new design becomes available. Philippe warned that they needed to be heated
before use. The new decoder card design itself still had to be done, but
should be easy - it just requires a new device in the library and re-layout.
There was no news from Sam on crates. On the power supplies, fives sets of
part had been ordered with a three week delivery time. Four were to be made
up and one set was for spares. One would be built at RAL, and three in
Birmingham.
The PDR for the PPM and PPr-ROD had taken place on 8th March in Heidelberg. PPM
specification draft 0.4 was reviewed for this module-0 design, and this included
the first draft of the line-receiver daughter board (AnIn). The PPr-ROD is a module
-1 design, and draft 0.1 of the specifications was reviewed.
The questions of the reviewers and others had been answered in comprehensive
and well organised written responses, which helped the review to proceed quickly
and amicably. Despite this, Tony felt in retrospect that although the PPM was
covered well, there really was not enough time to fully cover the PPr-ROD in
the same day. The general conclusions were that the PPM is now a well-specified
design, although a lot of the functionality itself has been reviewed separately.
The PPr-ROD and some of the components of the overall system (eg the Pipeline
Bus) need more description and possibly separate reviews.
Tony listed more detailed conclusions for both the PPM and PPr-ROD modules
which can be read in the transparencies. On the PPM, the most crucial of these
was the grouping of BC-Muxed pairs incorrectly in eta, but this could be fixed
by re-routing signals. Uli had some issues with the FCAL signal fanout, but
this has since been resolved between Uli and Paul. On the PPr-ROD, the main
area of contention was the Pipeline Bus and the complexity of the programming
model. It was asked that more explanation of the design should be provided.
The other review that had taken place was the FDR for the Pre-processor Multi-Chip
Module (PPrMCM). This was held on 12th April, using video and audio conferencing
which worked reasonable well. The document reviewed was draft 0.1 written by
Ullrich, whose imminent departure prompted the need for the FDR now. Due to
problems finding a suitable reviewer from ATLAS Technical Coordination, Philippe
was drafted in as the ATLAS panelist.
The document was an update on the PDR document from early 2000, and all of
the PDR recommendations had been addressed. The test program was still evolving
and good relations needed to be maintained with the production company, especially
with Ullrich soon leaving. There was a concern about failure rates, with a worryingly
high estimate given at one point, but real rates could not be known until the
chip went into production. The details are again listed in the transparencies.
Tony commented that the participation of Philippe was very important, and he
could again be helpful for the PRRs.
Nothing had changed significantly from previous reports except that
we are running a little later again. It was becoming clearer that
the UK modules would not be ready for a slice test before the end of
the year.
Testing and Integration
Dave reported that he had recently made a breakthrough and could now
generate and receive CAN frames with FADC voltages. He was continuing
to work on tidying up the libraries, and trying to use remote frame
requests. Future plans include investigating CANOPEN and SMBus, and
trying to integrate all the features. Dave agreed that he would write
an API description of his libraries, but right now most of the routines
are empty.
Norman firstly thanked Bruce for his enormous effort in making the tests
possible, and also Viraj and James for their swift backup. Pictures of
the integration test setup were presented, showing the main data paths -
data from the DSS went via a G-link to the ROD, which processed and output
the data onto two S-Links, which at various times were connected to the
RoI builder, DSS and ODIN cards.
Norman summarized the results - many of the objectives were fully or
partially achieved, with the main failure being problems at high speeds.
However at low speeds, long runs with completely verified data were possible -
these were actually better tests than had been achieved with the Muon system,
in that different events were sent and received correctly, whereas the muon
system only ever repeated the same event. Norman was keen to try a joint test
with the muon system, but it appeared that their system was not ready at this
time.
Many scope pictures were shown, all of which supported the correct operation
of the system. Latencies were stable and in most cases predictable in terms
of the amount of data transferred or simpler timing issues. One problem
observed via the scope was an occasional G-Link glitch which needs investigation.
The main problem area came with S-Link flow control between the ROD and RoIB.
Essentially there appeared to be synchronisation and buffering problems at
both ends - new firmware improved the situation at the ROD end, but it is
still not perfect. This problem meant that the system would hang at high speeds
when the buffering system started to be needed. Careful choice of trigger rates,
and bursts of triggers, was needed to avoid going into the flow control regime.
ROS tests were performed with the ODIN optical S-link. Continuous running up
to 20kHz were achieved, with higher rates possible in short bursts. The muon
system had managed to get to 40kHz, and it was not understood why we could not
do the same. A joint test with ROS and RoIB reading data was attempted, and
these ran for a over a million events before an error was found. Note that the
ROS software used was just a simple test program and we still have no experience
with the full ROS software.
Norman presented a long list of lessons learned from the integration test, which
can be found in the transparencies. Probably another month's work was needed to
solve most of the problems found. Norman concluded that the test setup at
CERN was very good, and the people helpful. He felt that the experience was
very helpful - it detected many errors that may otherwise not have been noticed.
Further integration with the muon system and new RoIBs would be desirable.
Software Status
News from the T/DAQ week was that new releases of software were
continuing to be rolled out. Of specific interest, new event dump
code should be available in June, and the TileCal presented their
new monitoring tool. ROS software was now available for one S-link
and should be extended to several S-links soon. There were also some
organisational changes affecting ROD crate DAQ. There was interest
in trying to better coordinate software across level-1, and a
video-conference was suggested in May/June.
Murrough reported on the current status of online software. Bill has
been ill, and Bruce had been mainly focussed on the ROD tests, so
the main progress had been Murrough's proposals for changes to HDMC
and integration of the HDMC hardware access into the DAQ system. A
document exists proposing changes to HDMC in order to better meet our
DAQ requirements, it can be found at:
http://www.hep.ph.qmw.ac.uk/l1calo/doc/pdf/HdmcChanges.pdf
Murrough had also redesigned the software website, now maintained under CVS in
order that everyone can modify the site. This can be found at:
http://www.hep.ph.qmw.ac.uk/l1calo/sweb
This contains much documentation of ideas for the slice test which people are
encouraged to read and comment on.
On hardware, three more Concurrent CPUs had been ordered and had in fact been
delivered to RAL. However, there is a newer version with better bus error
handling which could be an advantage to us, although this version has no
SCSI capability. Norman has been reading the documentation, and he and
Bruce will make a decision over which CPU to buy. Norman commented that
we need to understand the situation better as strange bus error behaviour
was encountered at the ROD tests. Murrough was keen that we have a closer
look at PC-ROS when we have obtained the necessary hardware. On this subject
Norman pointed out how the DAQ software was still in a state of flux as we
have never fully decided how it should be done - most of the ROD work at the
integration test was unfortunately done with temporary ad-hoc software.
Ed reported that the Em/Tau code was now written, with thresholds
being set via global values read from a text file. The code has
successfully generated RoI data, and will soon be tested on Alan's
test vectors. After that, he plans to start on the Jet/Et trigger
code.
Other Items
The tower summation problem may have returned, after apparently being resolved
at the ATLAS week in February. Rupert Leitner was still worried about the effect
on the Tau trigger. More discussion occurred at the T/DAQ week, and it now appears
that the proposal still needs approval at a TileCal week in May. It is by no means
guaranteed that we will get the option that we want, although one of the solutions
may be to provide both our preferred summing scheme, and the capability to switch
the other scheme. The LAr receiver stations had been reviewed on April 3rd, but
Eric had not yet heard any news of it.
Thoughts of ordering G-links for the whole system before/in case they go out of
production had given rise to a re-think of ROD topology. G-link transmitter chip
numbers are simply fixed by the CPM, CMM and JEM modules, but receiver numbers
are not so obvious.
The
currently documented and priced scheme requires 20 RODs with 16 G-links
per ROD. However many of these G-links are only needed for ROD
exchangeability, and not actually used. Norman has taken a more careful
look at data rates for this proposal - details can be found in the
transparencies. There were some inconsistencies in the original plans,
for example CMM readout had been forgotten. Note that the cost of G-links
is about £800 per ROD.
Depending on assumptions of how much data will be
recorded (for example number of slices), Norman calculated that the
maximum number of G-links needed would in fact be 17 for the Jet/Energy
RoI ROD, and 18 RODs would be needed. Only 190 out of the total of
306 G-links would be required and powered. This configuration could
just fit into one VME crate, but would probably be very uncomfortable, both
on the crate and the front panel (two ODINs are needed per ROD for output).
Norman presented an alternative scheme with denser RODs using four ODIN
outputs and each servicing a whole CP/JE crate DAQ output, rather than
half a crate in the original proposal. These would require 18 G-link
receivers each for the Jet/Energy crate. This would result in far fewer
G-links - 216 of which 190 are powered. The reduction to 12 RODs would
mean a far easier fit into one crate if they could still be single width
modules. The main problem is that the front panel would be even more
crowded, with no room for all the ODIN cards so they would have to be
mounted directly onto the ROD, and signals brought to the front panel.
Another issue here is that some of the modules would require the slower
ODINs, so therefore plug-ability of different types is required.
Decisions will have to be made soon to determine the G-link order, although
it was suggested we could buy enough for the dense option now, and more
later if needed.
Agreement with level-2 about ODIN strategy and with Sam and Uli must
be sought (possibly by phone conference). Studies of engineering
feasibility should also be done before any commitment to one solution.
Norman will present these ideas again at Mainz, with some proposals for
a cost sharing scheme.
A general discussion ensued about the relative merits of having one or two
ROD crates. It was felt that the extra expense of a second crate and CPU was
not significant compared to the difficulty of having a very full crate, especially
as there is already other ROD information coming from eight other (PPM) crates,
and more crates would be needing for test setups. The denser RODs do give a
large saving on G-link expense, but this is also probably not very significant.
Perhaps the 'modularity' of the RODs was a better criterion for a decision.
It was felt that it would be nice not to have to worry about the two S-link
speed options.
The IEEE deadline was imminent, and LEB the end of April. It was thought
that there was nothing good enough for the IEEE at present, but ideas for
LEB papers were one on the CMM design to illustrate FPGA versatility, and
one on the ROD status. Abstracts should be circulated soon to be checked
before submission. The muon RoIB tests had been written up as a paper, and
it was suggested Norman should do the same for the ROD integration tests -
the author list would have to be thought about.
There were no other issues raised.
Tuesday 5th June .............. UK meeting at RAL
28th - 30th June .............. Joint meeting at Mainz
Tuesday 17th July .............. UK meeting at Birmingham
|
The ATLAS Level-1 Calorimeter Trigger