Calibration Task List

maintained by Mat Martin
Last updated 3/11/04
Description Time Pr Package Schedule Person Status Details








Automate the table space allocation for defining a table
 2wk
 1
 DBTemplates
Mat(10%)
Currently the table-space for a table has to be hacked by hand. Using the rowcount.
Fill SET_SET_MAPS 10d 1 CalibDB ASAP Mat
It is not possible to trace which valid sets were used to create a daughter valid set. For example, PROD_PHYSICS_CDF is built out of several valid sets including PROD_PHYSICS_SVX. It is not possible to query which PROD_PHYSICS_CDF valid sets were based on a particular jobset for PROD_PHYSICS_SVX. This is because the there is no record of parentage of valid_sets in the database. A table to contain this informations exists, but the current API does not fill it. The table is called SET_SET_MAPS (jobset <-> jobset mapping like SET_RUN_MAPS which is a jobset <-> CID mapping). If this were filled then it would be possible to trace this relationship. This probably means adding a container of 'parent' jobsets to the ValidSet class in the same way that the PARENTS table is used for entries in calibrunlists (CID <-> CID mapping). 
Calib DB documentation
 -
 5
 CalibDB
 -
 Student
 -
 Add more documentation to CalibDB page:
   - A list of tables with clickable cdf-note (use Randy's browser)
   -
   
Regularize case in CodeGen 2-4wk 2 CodeGen ASAP
Check with Dennis to see if committed

Also delete REad only access
 Dennis
The current implementation of CodeGen allows database access methods which differ only in upper case and lower case, which then in Oracle refer to exactly the same table. This should be regularized to not allow such methods which differ only in upper case and lower case.
Replace PASSCALIBS with PASS_USED_MAPS 2-4wk 3 CalibDB 
FrameMods		 ASAP

Do a pl/sql test of 1000 such queries and time it.
 Mat
The current implicit connection between PASSES and USED_SETS using PASSCALIBS is not efficient for queries going from USED_SETS to PASSES. An explicit relationship implemented using a PASS_USED_MAPS table should solve this problem. This needs to be tested in integration and then changes to the perl script used to create passes and the CalibrationManager which reads passes would need to be done.
Multi run fetch ? 4
 CalibDB 
DBObjects 
DBManager		 needs full spec Mat,Dennis Deferred. 
To be reconsidered 
based on tests with used sets		 Sometimes the calibrations are small, e.g. beamlines, but change frequently. It would be good if there were a method to do a bulk fetch at the beginning for a group of runs.
Fetch by Process Name 2wk, needs full spec 4 CalibDB need input from willis sakamoto
 Mat
The API supports 2 fetching mechanisms. 
a) default fetch (get from global set, e.g. PROD_PHYSICS_CDF) 
b) runlist keyed get (get for run, version for that calibration) 

- There are times when the real question for analysis of data or 
calibrations is something like 'What did Production use for calibration A, and what did Level3 use?' In effect, you want to bypass the global used set, but you don't know the run, version to do a runlist keyed get. What you want is something that lets you specify a process name and get the calibration from that set. This would probably look like the current get based on RunListKey (b), but instead used something like a UsedSetKey to add a (c) to the list above. 
- Once SET_SET_MAPS is filled, it would even be possible for the global used set (e.g. PROD_PHYSICS_CDF) to point to a group of other valid sets (e.g. PROD_PHYSICS_SVX, FIB_PHYSICS_SVX) which could be fetches by process name based on global used set. 
- First implement fetch procedure based on process name alone. 
- Second consider procedure for using SET_SET_MAPS. 
- This would be much easier after the codegen re-write.
Bug in Used Set Fetch 3d 3
 CalibDB ASAP

send email to jack to see if this is done

 Jack Done for Oracle. 
In nightly validation		 Need support for fetch of Used Sets based on jobset number
Latest Run/Jobset for consumers 1wk 4
 FrameMods Talk to bill badget to see what his need is (see if we can get away with a pl/sql script)
 Jack(5%)/ 
Bill(0%)
For the online consumers, one needs to be able to compare to the valid set used by the last physics run or the latest jobset for a given process_name. This is not supported by the current CalibrationManager which assumes that you want data for the run being analyzed or for a specific jobset. 
- Can this be done by the CalibrationManager? 
- Should a script feed the tcl generator based on a query?








Storable Calibrations  1wk 2
 CalibDB 
DBTemplatesNew		 5.x Dimitri(5%)/Mat(5%)
Sometimes one wants to re-run reconstruction with simply the same calibrations used before. In this case it makes sense to keep those calibrations with the events to avoid unnecessary database connections. In order to do this one needs calibration with StorableBank implementations. 
- May be simple once new codegen is in place.
- Calibration manager has to have a switch which causes IOPackage to look in the beginRun record before it looks in the database
Update utilities

1

Mat
 DBUsedSet_multi written (a subset of the functionality of DBSetMerge)
 Provide generic tools for making ValidSets and usedsets etc to the detector groups.
















Run ranges for Dead/Hot channels (New, Improved) 2wk 3 CalibDB


Detector groups will write dead channels marked with the first run for which they are valid. These need to be grouped together for the various pieces of the detector and built into used sets.








Commit Calib Sim
1

Mat
Make Calib Sim runnable from the repository