Level 3 Online Status Reports
Back to index
Previous
Next
Issue 14
Thursday, September 30, 1999
Event Builder
Reports
- The ATM switch now contains two new switch fabrics (the old one is kept
as backup), running ForeThought 6.0. Routing cells between the two
fabrics requires the use of the expanded VPI field of internetwork-type
ATM cell headers. The expanded VPI encodes both the input and output
VPI's, which then enables us to keep our previous addressing scheme, by
which a transmitted ATM cell's VPI contains the destination id, and a
received cell's VPI contains the source id (thus enabling the receiving
hardware the determine whether or not it should accept the cell).
However, the price we pay in preserving this scheme is that all the
SCPU's have to be connected to one fabric while all the converters have
to be connected to the other. Steve has pre-assigned all the ports and
VPI's to nodes we know we'll have.
- The ATM switch fabrics have been moved from the public network to the
private network. The host tables on the PC's have been changed
accordingly.
- The ATM driver has been modified to report CRC errors in
/var/log/messages.
- It was found that all cells with VCI = 3 were lost in the switch.
Subsequent communications with FORE engineers revealed that, in fact,
all VCI's from 0 to 31 are reserved by the ATM standards committee
for special purposes; the event builder identifies the converter's
builder with the VCI, which puts us in exactly that low range. Our
current workaround is to patch scpu and l3recv code to use VCI's
modulo 3 and run the system with only 3 builders per converter.
Now event drops are rare and often in the first few events.
- The ATM adapters on the converter nodes have limited VPI+VCI
configurations. Currently we're using the configuration with 4 VCI bits
and 8 VPI bits. The next smaller VPI size is 2 bits, with 10 bits of
VCI space.
Steve tested a scheme by which 32 is added to the VCI on the SCPU
side, in the hopes that that receiving adapter will simply ignore those
higher bits and return the VCI modulo 16. It doesn't work; the
receiving adapter simply ignores those cells.
We may need another addressing scheme, this time using the VCI more.
- b0l3c06 may have a bad PCI bus which garbles SCRAMNet messages.
We may swap this node out and use it instead as a gateway node.
- it is noted that SVX calibration may require us to read out pedestals
for every single SVX strip, which would exceed the capacity of an
AAL5 packet. We may therefore need to add features to seqio_atm to
send larger packets (presumably by breaking the packet into a more
manageable size). Another alternative (not decided yet within the
SVX group) is to do the calibration processing on the SVX monitoring
VME CPU's, which can then be read out by the SCPU.
- Proxy status:
- new Java UCI: working since summer.
- message forwarder: needs some additional features, but basically
working.
- status pusher: not yet done.
- expert operations: scanner manager part finished and tested with toy
server; SCPU part not implemented yet.
- Event builder tests: current tests use up to 14 SCPU's and 8 converters.
Mean fragment sizes are 16K, and random variations are uncorrelated with
one another by choosing different random number seeds.
- N->1 tests:
- fixed-size fragments: event rate constant over N.
- 25% variation: slight degradation with increasing N. This effect
has been seen before, at which time it was attributed (though the
connection was not proven) to the fact that the average maximum
fragment size (which determines the transmission time for the
entire event) increases with N. However, Ilya points out that the
average maximum fragment size increases at a greater rate than the
drop observed; on the other hand, other latencies in the system
may suppess the effect. It is not clear that the earlier
attribution is correct.
- 4->N tests:
- fixed-size fragments; event rate is mostly constant, but slight
drops are observed at N=4 and N=7, with a larger drop at N=8. The
event rate was also observed to be roughly constant over time
(as opposed to systems with processor nodes, where the event rate
changes over time in a periodic pattern). It is not clear what
this drop is, or whether it could be a resonance effect. Obtaining
measurements with larger N is needed.
- 25% variation: there is a slight degradation with increasing N.
The "structure" (if that's what it is) observed above is not seen.
It is noted that when operating with up to 4 VME CPU receivers, the
event throughput was constant over N. It is not known at this point
why we now see a degradation.
Actions
- Steve, Ilya, and Sasha will continue on proxy work.
- Ilya will test multiple VRB readout and systems with 15 SCPU's and
more converter nodes.
Level 3
Reports
- Relay object: fires up other processes on remote nodes.
- Status readout code: needed some enhancements to Mike's history
object code; also needs some cleanup, but it basically works.
- Converter code: input and output modes should now be able to handle
multithreaded access. The analysis chain is to be made into a worker
thread, at which point the main thread becomes an overall monitor.
- Andreas has been investigating linuxconf for the management of
configuration files and daemons.
Actions
- Ivan will add "logical node group" capability to relay code.
- Andreas will refine status readout and history object code.
- Jeff will make the analysis chain a separate thread in the converter node.
Infrastructure
Reports
- Power is off on the third floor to rewire all the subfloor power lines.
In the meantime, we can move things around so as to reduce future
downtime.
- Central network rack: two Ethernet switches installed. One is
currently used to connect the active converter nodes with single
links. These are "unofficial" patch cords which are to be replaced.
- ATM rack: scanner manager (b0eb10) crate and ATM switch installed.
No SCRAMNet connection yet. The fiber bundle from the 1st floor has
been moved into location, with the ends coiled up underneath the
switch. They will be connected once we sort out which SCPU's will
be moved to the 1st floor.
- Converter shelf: 15 converters, of which 8 are currently active.
The are currently powered from the strips in the ATM rack, but this
will be changed to floor power once the right outlets are installed.
Their ATM fibers are coiled up on the shelf, which we'll leave until
the electricians have done their work.
- Output shelf: currently hold the new processor nodes. At present,
none are connected as we've been working with the converter nodes.
The old converter nodes will move here and become output nodes.
- Processor shelf: this is currently out of place by the wall. Dervin
has to notch a plain floor tile properly to accomodate the Ethernet
box mounted on the wall, at which point we can move all the ventilated
tiles in row 19 (between the two current shelf rows) and then move
the shelf into place. At this point, we move all the old processor
nodes, and Jim can have the old racks for his stuff (this will free
three racks, but he needs four; the 4th will come when we move the
SCPU's to the 1st floor).
- The Ethernet hookups in row 18 may be usable for the EVB/L3 network.
Check with JJ about this. The scanner manager and all the SCPU's also
need AUI-RJ45 adapters. JJ says DataComm should have lots of these
which we can get "cheap."
- Other cleanup: Jim has moved the PC's in row 17 elsewhere, so they
won't be in our way. Also, when we move more SCPU's to the 1st floor,
we'll be able to see how much of the rat's nest of optical fibers
we still need and act accordingly.
- Interfloor SCRAMNet feeds: we have one good duplex running from the
1st to 3rd floor. Ivan has tested the 5 long (120') duplexes for
use here; 2 will be used to run between the 1st floor and the 2nd-floor
TS crate, and the TS crate to the 3rd floor.
- 1st floor racks: Fred Lewis informed us that there is power of one
form or another in all of the VRB racks.
- rack 18I has a production Model 3 power supply, which
provides all the power lines we need for the 2600.
- 4 racks (21F, 21G, 21H, 21I) have modified Fastbus power supplies
which will supply all the power lines we need. These will be
replaced with production Model 3 power supplies.
- 3 racks (21C, 21D, 21E) have Model 1 power supplies which require
us to use special modified 9U adapters for the 2600's. These
crates already have these adapters in them, waiting for SCPU's.
21F-21I top crates are already filled with SCPU's. We will next
try to fill 21C-E, in which case we also need 6 9U adapters.
- Fred Lewis has temporary custody of b0eb11 in order to see what can
be done with the normal CDF 9U adapters to make them fit.
- All the Ethernet patch cords have arrived.
Actions
- Jeff will make sure floor tiles are moved and processor shelf located
correctly.
- Ilya will move as many SCPU's as we can to the 1st floor and hook them up
SCRAMNet and Ethernet.
- Steve will make sure we have two IP addresses for the
L3 gateway nodes.
- Jeff will add entries for the cable database
for SCRAMNet fibers and Ethernet.
- Jeff will check with JJ about Ethernet hookups and AUI-RJ45 adapters.
Back to index
Previous
Next
Jeff Tseng / MIT /
jtseng@fnal.gov