24/25 January 2002 ICRB meeting minutes

• We are moving very fast to an effective CDF GRID. There will be computers in various places that we will want to interconnect quite tighlty. The success of this will be based on
  • networks
  • software tools
  both we do not have now. Current work on software is moving in this direction.

• Several institutions have now a common plan: small farms in the trailers with a few TB disk and at least one dual PC per physicist. Notable exceptions are, of course, overseas: UK, Italy, Korea, Spain are all planning (with somehow different time scales) to have large farms over there. There are also US institutions with lots of computers at home.
  There are at present mixed feelings about pooling CDF hardware toghether as a shared resource across institutions. Some are already doing it, some are considering it, some are against. Some intitutions who have no plan to pool hardware resources with FCC ones are willing to think it over once a clear plan is layed out and advantages (if any) are made apparent.
  This situation may thus evolve once the new CAF is in place and data flows are better understood. Having the hardware in the trailers offers the clear advantage of resource control, while putting it at FCC offers professional opration/mainteance and possibly faster access to the tape robots, besides freeing physicists from needing to be part-time sys.manager and some noise/heat considerations, but the hardware has to become at least partially of common usage. An example of such an arrangement is the Italian disk at FCC.

• Expectations for data copying offsite are large and can only be satisfied with a significant upgrade of the Fermilab connectivity. It is very unlikley that a common tape copy facillity could be setup to provide similar capacity, so in the lack of better network people may have to resort to self-managed tapes in the trailers.

• It is the chair's opinion that CDF must move relentlessy and aggressively to obain the needed network upgrades to avoid the Fnal/world connection to become a bottleneck. CDF analysis is very likley to be resource-limited in the coming years and logistic imposes a limit to what can be done at FNAL. Being able to transparenlty move data and/or jobs to other places may be the only way to effetively add manpower to hardware installation/operation/maintenance. Besides all the usual GRID propaganda, and besides the fact that all in all Fnal may not be able to buy al the hardware that we may want here and especially other countries will have much much more computing power at home then they will ever add to FCC.

• Next meeting will not overlap with collaboration meeting.

• kernel troubles Getting the newest hardware to work is not straightforward.
  • several people spent lot of time dealing with or worrying about hardware issues for Linux compatibility
  • reliability, robustness
  • problems with Linux kernel
  • all agree on the the idea of "certified" configurations developed jonlty and made available
• ftp Several people reported abysmal performance of network copy from FNAL to offsite (~1Mbit/sec) in spite of much larger nominal network bandwidth, and regardless of distance from Fnal (e.g. Duke and Italy)
  • possibly due to "known" problems with ftp buffer size which should be addresses by "GRID tools" like bbftp (which is e.g. used in D0/CD product SAM or GRIDftp
  • or simply fixed by many users copying in parallel
  • or one user making many parallel copies
• Fnal version numbers At least some problems to make FERMI software accepted/used offsite are due to version numbering
  • FermiRH7.1 is actually almost a full RH7.2 in spite of the name.
  • Fermi ssh v1.2.27g is NOT the dreaded 1.2.27 at all
• OpenSsh The lab is working to move to OpenSsh, but due to limited manpower it will "months". In the meanwhile it appears that all issues of inter-operatibility with Ssh have viable workaround.
• info for sysman Information about problems (and solutions!) for system managers of computers where CDF software is installed should be shared/spreaded more effectively. A mailing list for this already exists and people should use it.
  • list name is cdf_hosts
  • you can send e-mail to cdf_hosts@fnal.gov
  • you can (un)subscribe by sending e-mail to listserv@fnal.gov with no subject and the text
    (un)subscribe cdf_hosts
  • Art Kreymer is the list owner
• Making ntuple It appears that now is more effective to copy files to the trailers and make the ntuple there, rather then making the ntuple on fcdfsgi2 and simply copying that
• tape copy An informal survey of present institutions (about 10) showed large interest for copying large data sets offsite, it is very unrealistic that a SONY AIT-2 based copy facility could satisfy that demand, it would require tens of drives running continously with the same operational headaches that made us switch to STK for the experiment Data Handling system.

• RedHat 7.2 CDF code works on RedHat 7.2 (called FermiLinux 7.1, see above about numberings) starting with version 4.2.0. See Art's log or Italy's log. The code management group looks forward to being able to drop support for RedHat 6 to diminuish their work load. Therefore everybody is encouraged to migrate to RedHat 7 if convenient. On the other hand Level 3 and production farm currently run RedHat 6, so there is no saying when dropping support for RH6 may be considered. Also Fermilab expert Connie Sieh advice that some older hardware is better off with RedHat 6. In particular machines with Adaptec SCSI interfaces, Connie claims their performance under RedHat 7 is presently poor.
• gcc It compiles, but it does not link. The person working on this is phasing out (see Kevin's talk at collab.meeting). It will have to be picked up by interested institutions. Libraries are about 8x larger. Nothing is known about executable sizes and/or speed until an image can be linked.