Level 3 Online Status Reports

Back to index Previous Next

Issue 14
Thursday, September 30, 1999

Event Builder

Reports

• The ATM switch now contains two new switch fabrics (the old one is kept as backup), running ForeThought 6.0. Routing cells between the two fabrics requires the use of the expanded VPI field of internetwork-type ATM cell headers. The expanded VPI encodes both the input and output VPI's, which then enables us to keep our previous addressing scheme, by which a transmitted ATM cell's VPI contains the destination id, and a received cell's VPI contains the source id (thus enabling the receiving hardware the determine whether or not it should accept the cell). However, the price we pay in preserving this scheme is that all the SCPU's have to be connected to one fabric while all the converters have to be connected to the other. Steve has pre-assigned all the ports and VPI's to nodes we know we'll have.
• The ATM switch fabrics have been moved from the public network to the private network. The host tables on the PC's have been changed accordingly.
• The ATM driver has been modified to report CRC errors in /var/log/messages.
• It was found that all cells with VCI = 3 were lost in the switch. Subsequent communications with FORE engineers revealed that, in fact, all VCI's from 0 to 31 are reserved by the ATM standards committee for special purposes; the event builder identifies the converter's builder with the VCI, which puts us in exactly that low range. Our current workaround is to patch scpu and l3recv code to use VCI's modulo 3 and run the system with only 3 builders per converter. Now event drops are rare and often in the first few events.
• The ATM adapters on the converter nodes have limited VPI+VCI configurations. Currently we're using the configuration with 4 VCI bits and 8 VPI bits. The next smaller VPI size is 2 bits, with 10 bits of VCI space. Steve tested a scheme by which 32 is added to the VCI on the SCPU side, in the hopes that that receiving adapter will simply ignore those higher bits and return the VCI modulo 16. It doesn't work; the receiving adapter simply ignores those cells. We may need another addressing scheme, this time using the VCI more.
• b0l3c06 may have a bad PCI bus which garbles SCRAMNet messages. We may swap this node out and use it instead as a gateway node.
• it is noted that SVX calibration may require us to read out pedestals for every single SVX strip, which would exceed the capacity of an AAL5 packet. We may therefore need to add features to seqio_atm to send larger packets (presumably by breaking the packet into a more manageable size). Another alternative (not decided yet within the SVX group) is to do the calibration processing on the SVX monitoring VME CPU's, which can then be read out by the SCPU.
• Proxy status:
  • new Java UCI: working since summer.
  • message forwarder: needs some additional features, but basically working.
  • status pusher: not yet done.
  • expert operations: scanner manager part finished and tested with toy server; SCPU part not implemented yet.
• Event builder tests: current tests use up to 14 SCPU's and 8 converters. Mean fragment sizes are 16K, and random variations are uncorrelated with one another by choosing different random number seeds.
  • N->1 tests:
  • fixed-size fragments: event rate constant over N.
  • 25% variation: slight degradation with increasing N. This effect has been seen before, at which time it was attributed (though the connection was not proven) to the fact that the average maximum fragment size (which determines the transmission time for the entire event) increases with N. However, Ilya points out that the average maximum fragment size increases at a greater rate than the drop observed; on the other hand, other latencies in the system may suppess the effect. It is not clear that the earlier attribution is correct.
  • 4->N tests:
    • fixed-size fragments; event rate is mostly constant, but slight drops are observed at N=4 and N=7, with a larger drop at N=8. The event rate was also observed to be roughly constant over time (as opposed to systems with processor nodes, where the event rate changes over time in a periodic pattern). It is not clear what this drop is, or whether it could be a resonance effect. Obtaining measurements with larger N is needed.
    • 25% variation: there is a slight degradation with increasing N.
    The "structure" (if that's what it is) observed above is not seen. It is noted that when operating with up to 4 VME CPU receivers, the event throughput was constant over N. It is not known at this point why we now see a degradation.

Actions

• Steve, Ilya, and Sasha will continue on proxy work.
• Ilya will test multiple VRB readout and systems with 15 SCPU's and more converter nodes.

Level 3

Reports

• Relay object: fires up other processes on remote nodes.
• Status readout code: needed some enhancements to Mike's history object code; also needs some cleanup, but it basically works.
• Converter code: input and output modes should now be able to handle multithreaded access. The analysis chain is to be made into a worker thread, at which point the main thread becomes an overall monitor.
• Andreas has been investigating linuxconf for the management of configuration files and daemons.

Actions

• Ivan will add "logical node group" capability to relay code.
• Andreas will refine status readout and history object code.
• Jeff will make the analysis chain a separate thread in the converter node.

Infrastructure

Reports

• Power is off on the third floor to rewire all the subfloor power lines. In the meantime, we can move things around so as to reduce future downtime.
• Central network rack: two Ethernet switches installed. One is currently used to connect the active converter nodes with single links. These are "unofficial" patch cords which are to be replaced.
• ATM rack: scanner manager (b0eb10) crate and ATM switch installed. No SCRAMNet connection yet. The fiber bundle from the 1st floor has been moved into location, with the ends coiled up underneath the switch. They will be connected once we sort out which SCPU's will be moved to the 1st floor.
• Converter shelf: 15 converters, of which 8 are currently active. The are currently powered from the strips in the ATM rack, but this will be changed to floor power once the right outlets are installed. Their ATM fibers are coiled up on the shelf, which we'll leave until the electricians have done their work.
• Output shelf: currently hold the new processor nodes. At present, none are connected as we've been working with the converter nodes. The old converter nodes will move here and become output nodes.
• Processor shelf: this is currently out of place by the wall. Dervin has to notch a plain floor tile properly to accomodate the Ethernet box mounted on the wall, at which point we can move all the ventilated tiles in row 19 (between the two current shelf rows) and then move the shelf into place. At this point, we move all the old processor nodes, and Jim can have the old racks for his stuff (this will free three racks, but he needs four; the 4th will come when we move the SCPU's to the 1st floor).
• The Ethernet hookups in row 18 may be usable for the EVB/L3 network. Check with JJ about this. The scanner manager and all the SCPU's also need AUI-RJ45 adapters. JJ says DataComm should have lots of these which we can get "cheap."
• Other cleanup: Jim has moved the PC's in row 17 elsewhere, so they won't be in our way. Also, when we move more SCPU's to the 1st floor, we'll be able to see how much of the rat's nest of optical fibers we still need and act accordingly.
• Interfloor SCRAMNet feeds: we have one good duplex running from the 1st to 3rd floor. Ivan has tested the 5 long (120') duplexes for use here; 2 will be used to run between the 1st floor and the 2nd-floor TS crate, and the TS crate to the 3rd floor.
• 1st floor racks: Fred Lewis informed us that there is power of one form or another in all of the VRB racks.
  • rack 18I has a production Model 3 power supply, which provides all the power lines we need for the 2600.
  • 4 racks (21F, 21G, 21H, 21I) have modified Fastbus power supplies which will supply all the power lines we need. These will be replaced with production Model 3 power supplies.
  • 3 racks (21C, 21D, 21E) have Model 1 power supplies which require us to use special modified 9U adapters for the 2600's. These crates already have these adapters in them, waiting for SCPU's. 21F-21I top crates are already filled with SCPU's. We will next try to fill 21C-E, in which case we also need 6 9U adapters.
• Fred Lewis has temporary custody of b0eb11 in order to see what can be done with the normal CDF 9U adapters to make them fit.
• All the Ethernet patch cords have arrived.

Actions

• Jeff will make sure floor tiles are moved and processor shelf located correctly.
• Ilya will move as many SCPU's as we can to the 1st floor and hook them up SCRAMNet and Ethernet.
• Steve will make sure we have two IP addresses for the L3 gateway nodes.
• Jeff will add entries for the cable database for SCRAMNet fibers and Ethernet.
• Jeff will check with JJ about Ethernet hookups and AUI-RJ45 adapters.

Back to index Previous Next