DH system monitoring
  1. Check that raw data is being logged to tape. This can be done via the web:
    1. check that the data logger disks (disk3-disk9) aren't filling up. These plots show the disk space over time; there should be jagged edges where data is dumped to tape (about 50GB each time). The blue plot shows the disk space occupied by data, the violet plot shows the total disk used
      • reload the web page, then check time of the last update of the logger disk usage plot - it should be updated every 20 minutes.
        If the plots are out of date, report to SCICO and page the CSL expert
      • if the violet histogram is above 75% for any of disk3-disk9, report to SCICO and page the CSL expert
      • if the blue histogram is above 50% for any of disk3-disk9 report to SCICO and page the CSL expert
      • CSL group email list: cdf_csl@fnal.gov.

      Here are 2 examples of problems with the logger disks
      Problems with logger disks - example 1
      Problems with logger disks - example 2 (disk9)

    2. Check record of the raw data tapes written. New tape write errors (in red) should be reported to cdfdh_oper@fnal.gov. If in 12 hours cdfdh_oper@fnal.gov didn't reply, report the problem to cdf_offline_operations@fnal.gov.
for the purpose of editing: this file is located at
cdfsga:/cdf/www/upgrades/computing/operations/offline_shifts/data_handling_monitoring.html