Welcome to the Trybos On-line User's Guide. This document is intended to provide an informal introduction to Trybos, with many small code examples demonstrating common programming tasks involving the Trybos product. This guide treats Trybos version 3.3, which is not yet integrated into a cdfsoft2 release. Much of this document is also appropriate for Trybos version 3.2, which is integrated into cdfsoft2 releases 1.4.3 and 1.6.0.
Trybos is an Object-Oriented C++ package to read, write, and manipulate data in CDF YBOS format. While it is currently intended for use by C++ programs, it will eventually provide a substantial YBOS API emulation for FORTRAN-77 program use. Trybos is coded entirely in Standard C++ and uses no pre-existing CDF offline software. Files written by Trybos are indistinguishable from files written by the current CDF YBOS software package. Both Run 1A 'VMS byte-order' and Run 1B data 'Unix byte-order' data can be read, and 'Unix byte-order' data format can be written. Trybos has been verified to operate on at least the following Unix systems with GNU C++ compiler 2.7.2: SGI IRIX, Intel-based Linux, DEC Unix.
As in the past, a programmer in the Run II CDF Framework / Trybos software environment does not have to explicitly open and close files, or read and write events. These actions are performed for the user by the I/O modules provided with the Framework product. Most programmers will perform just a few operations with Trybos: locating banks in a record, accessing data in a bank, storing or removing a bank in a record, and creating a C++ class describing a bank.
Trybos contains a number of components which the general user should be familiar with in order to fully exploit Trybos. Many of these components influence the entire CDF Run II software environment.
The Trybos (and YBOS) data storage model is that of a one-dimensional array of four-byte, signed integers. Since C and C++ do not guarantee the exact size of integer types, the typedef "int4" is defined in the package header file TRY_Package.hh to describe a four-byte signed integer on all supported operating systems and compilers. Wherever in this text that a quantity is referred to an integer, it should be understood that it is a four-byte signed integer declared as an int4. Other fixed-size and related types include:
The "char" data type is used in Trybos in preference to the "int1" data type, and the "byte" data type is used in preference to the "uint1" data type.
There are four characters in a TRY_Bank_Name. Each character in the name must be printable. There are otherwise no restrictions on bank names. A bank name can be initialized or accessed as a string or as in integer.
There is no wildcard support in TRY_Bank_Name. Since there are now two record iterator classes which distinguish any-name and same-name iteration, a wildcard for TRY_Bank_Name is unnecessary and leads to less efficient name comparisons. Wildcarding of Bank Names will be supported with TRY_Bank_Name_Set (to be developed) which will act much like YBOS name sets and subsets.
TRY_Bank_Numbers are generally non-negative integers. They can also have one of several special values indicating the number will match any other number or no valid number, but these special values cannot be stored in a bank. There are otherwise no restrictions on bank numbers.
All banks in the YBOS format are identified by a (bank name, bank number) pair, which is encapsulated as TRY_Bank_Key in Trybos. Bank searches use the method TRY_Bank_Key::operator ==() to determine if two keys are considered a match.
Banks with last character in their name == "D" and with bank number == 100 are assumed to have their data in a "squeezed" state. This does not affect Trybos behavior at present, but it is an issue for backward compatibility with YBOS software and software associated with squeezed data formats.
In Trybos, unlike YBOS, the bank type information is treated as completely separate from the user data stored in a bank. The data types of user data stored in a bank are described by an instance of the class TRY_Bank_Type. Bank types can be initialized and accessed as either a string or an array. The string version of the Bank type representation follows a specific format which is similar to that used by YBOS's BTYMAK() subprogram. The data elements can be any one of the following supported type specifications:
Mistakes in some bank type specifications have led to banks with superfluous zero-valued words in the type specification. For the sake of completeness such null type words are indicated by the specification "??".
Banks consisting of only one data type are referred to as mono-type banks. Their data type are specified by a base-10 integer followed by a type specification, with no whitespace between the two. The leading integer is a repeat count. The repeat count and size of the data type must yield a data size which is a whole number of 4-byte integers. For example, one can declare a mono-type bank type to be 28AS (stored in 7 integers), but not 27AS.
Banks with more than one data type specified are referred to as mixed-type banks. Multiple mono-type specifications can be catenated with a comma. Repeated groups within a type specification can be grouped by parentheses and given a group count. Since any mixed-type bank is implicitly a grouping with count one, all mixed-type banks have outer parenthesis, but the outer group count is omitted since it is always exactly one. These outer parentheses are required to distinguish between a mono-type bank specification and a mixed-type bank specification with just one data type.
TRY_Generic_Bank provides all the common functionality YBOS banks. Banks can be stored in a record, located within a record, have data representation (byte order) automatically converted if necessary, and be printed in a generic fashion. TRY_Generic_Bank also provides protected data access facilities as tools for derived bank classes to use in implementation.
A TRY_Generic_Bank is considered to consist of three sections: header, type specification, and user data sections. Each of these have a distinct size which the programmer can access. The combination of the type specification and the user data sections are referred to as the bank body. This is slightly different from the YBOS picture where the bank size was the size of the bank body only, and in Trybos the bank size includes the size the bank header.
All specific (named) bank classes are derived from TRY_Generic_Bank in order to become a manipulatable YBOS bank. All software associated with specific bank classes is maintained in the Banks product. This includes the specific bank header files, the BNK_generate_headers source file, and any source files used to implement specific bank classes.
Users working in the Framework environment do not have to read/write records or manipulate files directly. In the event-processing part of a Framework module, the programmer is given a pointer to an TRY_Abstract_Record:
TRY_Abstract_Record *p_record ;
without having to select the exact implementation of the record, and without having to read an event into the record. The Framework I/O modules will select an implementation for the TRY_Abstract_Record pointer. All the user needs to know is that all operations on the record are performed by dereferencing the pointer. For instance, to append an LRID bank called "my_LRID_bank" to the event record (if it is called p_record), one would use:
p_record-append(my_LRID_bank) ;
Iterators are an abstraction for a pointer into a collection of objects. Iterator classes are usually designed so that iterators "know" the bounds and other information describing a collection. An advantage of iterators over simple pointers is that iterators can move from one object to another (iterate) in a collection of differently sized objects. Simple pointers can, in general, only treat collections of fixed-size objects.
Record Iterators are used to specify the location of a bank in a record. They may be though of as pointers to TRY_Generic_Banks, or as iterators into a TRY_Abstract_Record collection of TRY_Generic_Banks. In Trybos, a record iterator can be in one of two states: it can be pointing to a bank in a record, or it can be in an "invalid" state. Programmers should use the methods is_valid() or is_invalid() to test whether an iterator is valid before using the iterator.
TRY_Record_Iter_Any and TRY_Record_Iter_Same are the two record iterator classes in Trybos. The two classes are identical, except for the iteration behavior. In the class TRY_Record_Iter_Any, the operator ++() will cause the iterator to point to the next bank of any bank name or, if the end of the record is reached, cause the iterator to become invalid. In the class TRY_Record_Iter_Same, operator ++() will cause the iterator to point to the next bank of the same bank name, or cause the iterator to become invalid.
There are just a few steps to using record iteration. First, one must initialize the record iterator variable. This is often roughly equivalent to calling BLOCAT() when using YBOS. Then one should always test for validity if the iterator is expected to point to a bank, to be sure that it is in fact pointing to a bank. This is equivalent to testing the status code returned from BLOCAT() when using YBOS. Finally, one can attempt to find the next bank of any bank name (TRY_Record_Iter_Any) or of the same bank name (TRY_Record_Iter_Same) using one of the iteration methods. This is roughly equivalent to calling BNEXT() and BDATA() when using YBOS.
As simple as that may sound, a few example of common programming tasks involving record iterators may prove enlightening. There are afterall a number of means to accomplish each of these three steps depending on the task at hand.
In this example, the user wishes to locate a particular TRKS bank, with bank number 121. Here, it makes no difference whether an iter-any or an iter-same iterator is used. Note that a temporary TRY_Bank_Key variable is used in the second version to feed in the particular bank number desired.
TRY_Bank_Key my_key("TRKS",121) ;
TRY_Record_Iter_Any my_iter(p_record, my_key) ;
if (my_iter.is_valid())
{ // Succeeded in locating ("TRKS", 121)
}
And essentially the same code is used if one would prefer an iter-same and to use a temporary TRY_Bank_Key instead.
TRY_Record_Iter_Same my_iter(p_record, TRY_Bank_Key("TRKS",121)) ;
if (my_iter.is_valid())
{
// Succeeded in locating ("TRKS", 121)
}
In this example, the programmer wishes to loop through all the banks in a record, one at a time. The initialization clause assures that the iterator points to the first bank in the record, whatever the bank's name is. The end test clause checks that the iterator is still valid (not at end of record). If there are no banks in the record, this loop will not execute at all, as desired. Finally the iteration clause causes the iterator to point to the next bank, whatever the bank's name is. The banks are visited in the same order in which they happen to be arranged in the record. There is no guarantee that the bank names or numbers, for instance, will be in increasing or decreasing order.
// for loop initialization................ end test............ iteration
// -----------------------------------------------------------------------
for (TRY_Record_Iter_Any my_iter(p_record) ; my_iter.is_valid() ; ++my_iter)
{
// my_iter points to each bank in record sequentially
}
In this example, the programmer wishes to loop through all CMUO banks in a record, one at a time. The initialization clause assures that the iterator points to the first CMUO bank in the record. The end test clause checks that the iterator is still valid (not at end of record). If there are no CMUO Banks in the record, this loop will not execute at all, as desired. Finally the iteration clause causes the iterator to point to the next CMUO bank. The CMUO banks are visited in the same order in which they happen to be arranged in the record. There is no guarantee that the bank numbers, for instance, will be in increasing or decreasing order.
for (TRY_Record_Iter_Same my_iter(p_record,"CMUO") ;
my_iter.is_valid() ; ++my_iter)
{
// my_iter points to each CMUO bank in record sequentially
}
In this example, the programmer wishes to consider all unique pairs of CMUO banks found in a record. The outer loop is the same as in the previous example. The inner loop differs though. The inner loop's initialization clause uses the value corresponding to an record iterator pointing to the CMUO bank after the CMUO bank pointed to by my_iter1 to initialize iter2.
for (TRY_Record_Iter_Same my_iter1(p_record,"CMUO") ;
my_iter1.is_valid() ; ++my_iter1)
{
for (TRY_Record_Iter_Same my_iter2(my_iter1.copy_next()) ;
my_iter2.is_valid() ; ++my_iter2)
{
// my_iter1 points to each CMUO Bank in Record sequentially
// my_iter2 is initialized to the CMUO Bank after my_iter1 (if any)
// (my_iter1, my_iter2) cover all unique CMUO bank pairs in Record
}
}
for (TRY_Record_Iter_Same my_iter2(my_iter1), ++my_iter2 ;
my_iter2.is_valid() ; ++my_iter2)
The reason this fails is a language detail, but may be of interest to some. C++ compilers reject this because my_iter2 is not guaranteed to be constructed BEFORE it is iterated because the comma in C++ does not act as a sequence point, though a semi-colon does. On the other hand,
TRY_Record_Iter_Same my_iter2(my_iter1) ; // this semi-colon is a sequence point unlike comma ++my_iter2 ;is in fact equivalent to
TRY_Record_Iter_Same my_iter2(my_iter1.copy_next()) ;because of the intervening semi-colon. Hence, each record iterator class has a copy_next() method to deal with this and related loop scenarios where an iterated value is needed without affecting the original value.
The internal state of iterators on a record is ill-defined after record modifications. Thus, if one wishes to perform a record modification such as a insert() or erase(), then the iterator driving the loop may not be able to safely iterate itself on the next loop iteration. Once the Fortran API implementation for Trybos is better defined, safe iterators should be possible to be implemented. Until then, as with YBOS, the programmer should not expect record iterators to maintain their value after any record modifications occur.
In order to work around such limitations in the meantime, a number of additional components will be included in a near-future version of Trybos. Several methods will be provided to replace loops with common operations on all specific Banks in a record, such as TRY_Abstract_Record::remove(name). The remove() method will replace both the loop over banks and the erase() operation, erasing all Banks with the given name. Also, a class is planned for a future Trybos release to define sets of Bank families, TRY_Bank_Name_Set. More alternative forms of common methods will be provided which operate on banks with a name matching any name in given the name set.
The record iterator user interface as it exists now was selected over the STL style of iterator interfaces for several reasons. Tests of iterator validity are frequent, which can lead to noticeable CPU overhead if an iterator begin() or end() must be constructed for each such test. In addition, it is more difficult to maintain classes which return instances of other classes. Another consideration is that it is impossible in C++ to overload begin() on its return type, making it difficult to implement multiple begin() and end() methods in each Trybos record class for each record iterator class. Many CDF C++ programmers seem to find the iterator style used by Barton and Nackman more appealing (though perhaps that is simply because the book is more readable that many STL guides), Trybos has adapted a similar interface to meet FNAL and CDF C++ style guidelines.
All examples here use "pre-increment" instead of "post-increment", as in ++my_iter instead of my_iter++. Post-incrementing generally involves a little extra CPU time relative to pre-incrementing, unless the compiler's optimizer can optimize the implied CPU overhead away. Rather than depend on this behavior, all examples simply use pre-incrementing, unless post-incrementing is specifically required.
All specific (named) bank classes are derived from TRY_Generic_Bank in order to become a manipulatable YBOS bank. All bank classes consist of a header containing a valid bank key and a valid bank size, a valid type specification in array format, and a user data section. Since the Trybos/YBOS data format specification is so loose, allowing any number of data formats to be described, precise discussion of all specific bank classes is impossible. Examples of simpler specific bank classes are given as they are developed.
There are two basic methods for creating, copying, or assigning a bank in Trybos: copy_val() which performs a copy-by-value, and copy_ref() which performs a copy-by-reference. In a copy-by-value, memory is allocated for a new, distinct bank and the values from one bank are copied into the newly-allocated memory. In a copy-by-reference, no new memory is allocated; only the pointers to the information in one bank are copied to pointers to the same information in the new bank. In general, banks which are stored in a record (recorded banks) are manipulated with copy_ref(), and those which are not stored in a record (unrecorded banks) are manipulated by copy_val(). This is largely an optimization, but one which has a side-effects. If one alters the values in a recorded bank, one alters the values in the record. If one alters the values in an unrecorded bank, one only alters the values in that bank.
Consider an example of creating a fixed-size, mono-type bank, LRID_Bank, An An instance of this class can be constructed on the heap (an unrecorded bank) or inside a record (a recorded bank). Since there is no extra information required to determine the size of an LRID_Bank, the unrecorded bank constructor just takes the bank number as an argument, while the recorded bank constructor must also have the record involved as an argument.
explicit LRID_Bank(const TRY_Bank_Number& number) ; // New unrecorded Bank
explicit LRID_Bank(const TRY_Bank_Number& number, // New recorded Bank
const TRY_Abstract_Record *p_record) ;
Both constructors set the bank header and type information, but do not set any
default values for the user data section of the bank. Each of these
constructors would take more arguments if the bank size varied.
In addition to these, there are constructors which take record iterators as arguments in order to convert from iterators to a full-fledged bank. These constructors are the same for variable-sized banks.
explicit LRID_Bank(const TRY_Record_Iter_Any& my_iter) ; // == copy_ref() explicit LRID_Bank(const TRY_Record_Iter_Same& my_iter) ; // == copy_ref()
There are also constructors taking a generic bank or another LRID bank as arguments as well as an assignment operator. They do not change for variable-sized banks. The copy_val() and copy_rep() methods are used to avoid replicating code between several constructors and the assignment operator. The behavior of assign() varies according to whether the arguments are related to a recorded bank or not. Constructing a bank which is recorded, or initializing with a bank which is recorded, only causes a copy of the pointers to the data, not the data itself. Hence, any change to data values in such a bank object will cause the recorded bank data to also change. If the user wishes to force a particular behavior, alternatives to the assign method are also provided which specifically copy data into new memory (a "deep" copy) or simply copy pointers to the data (a "shallow" copy).
void copy_val(const LRID_Bank& my_bank) ; // copy by value void copy_ref(const LRID_Bank& my_bank) ; // copy by reference explicit LRID_Bank(const LRID_Bank& my_bank) ; // Copy constructor explicit LRID_Bank(const TRY_Generic_Bank& anon_bank) ; // Copy-like constructor LRID_Bank& operator =(const LRID_Bank& my_bank) ; // Assignment operator
Once a bank is created, it can be tested for validity or compared to another bank. Validity for different bank classes may be defined differently, though it should at least contain TRY_Generic_Bank::is_valid() in its definition. Comparisons are made between the bank header, type, and data values and are based on the member function compare().
bool is_valid(void) const ;
bool is_invalid(void) const ;
bool compare(const LRID_Bank& my_bank) const ;
friend bool operator == (const LRID_Bank& bank1,
const LRID_Bank& bank2) ;
friend bool operator != (const LRID_Bank& bank1,
const LRID_Bank& bank2) ;
TRY_Generic_Bank provides a selection of protected methods to simplify data access. Methods treat all fundamental YBOS data types with individual element access, except ASCII characters which are treated as part of a C++ string and double precision floating point which is not yet implemented.
byte get_BY_element(const int4 offset, const int4 index) const ; uint2 get_I2_element(const int4 offset, const int4 index) const ; int4 get_I4_element(const int4 offset, const int4 index) const ; float4 get_R4_element(const int4 offset, const int4 index) const ; bool set_BY_element(const int4 offset, const int4 index, const byte value) ; bool set_I2_element(const int4 offset, const int4 index, const uint2 value) ; bool set_I4_element(const int4 offset, const int4 index, const int4 value) ; bool set_R4_element(const int4 offset, const int4 index, const float4 value) ; string get_AS_string( const int4 offset, const int4 field_words) const ; bool set_AS_string( const int4 offset, const int4 field_words, const string& value) ;Since the different specific banks will implement different abstractions, however, not very much more can be said about data access in general. All specific banks classes that will be automatically generated will provide at least a raw level of read access to int4 array in which data is stored, regardless of type of data elements. No raw "set" methods are provided since this could lead to floating point and other exceptions if inappropriate values are installed in the storage used for non-integer data.
int4 n_raw_data_words(void) ; // number of int4 words used for data int4 raw_data_word(const int4 index) ; // 0 <= index < n_raw_data_words()
For mono-type and simple mixed-type banks, properly typed access to data elements will be provided in automatically generated bank classes via plain-name accessors. Examples of some such methods for different data types follow, each with an appropriate method to indicate the maximum value for the index:
int4 n_elements(void) const ; // number of data items stored in bank // AND ONE OF THE FOLLOWING ACCESSORS byte BY_element(const int4 index) ; uint2 I2_element(const int4 index) ; int4 I4_element(const int4 index) ; float4 R4_element(const int4 index) ; char AS_string( const int4 index) ; // AND ONE OF THE FOLLOWING MODIFIERS bool set_BY_element(const int4 index, const byte value) ; bool set_I2_element(const int4 index, const int2 value) ; bool set_I4_element(const int4 index, const int4 value) ; bool set_R4_element(const int4 index, const float4 value) ; bool set_AS_string(const string& value) ;
Some banks will require a significant number of constants to be defined in order to describe the data layout. These should be put into a file named XXXX_Bank_format.hh for bank class XXXX_Bank and included in the XXX_Bank.hh class header. LRID_Bank, for instance, has many bitfields defined in its data members which are described in LRID_Bank_format.hh.
There are several means of storing a bank in a record. As mentioned in a previous section, one can literally create a bank in a record. This is the most CPU efficient means of creating a bank provided the bank is intended to be in the record in the first place. Such a creation always involves appending the bank to the end of the record, just as the record method append() does.
The method insert() is provided in the record classes which provides the user with more control over where the bank goes into the record, though the method is less CPU-efficient than a simple append().
There are also several means of removing a bank from a record. The method erase() removes a single bank pointed to by its record iterator argument. The bank is not returned. The method remove() erases all banks matching its bank name argument from a record. Finally the new method extract() is identical to erase() except that the erased bank is returned to the user.
The program BNK_generate_headers treats fixed-size mono-type banks, variable-sized mono-type banks, and flat mixed-type banks found in datafiles. It generates a boilerplate C++ header file for each acceptable bank name it finds. Experts in each bank's functionality will still want to add named accessor methods to these classes after they are produced, but much of the tedious work can at least be done automatically. BNK_generate_banks currently has produces class headers (which may or may not provide the appropriate abstraction for any particular bank) for the following banks:
$3CM $CLP $CPO $DPC $DPL $MBC $SCF $SVC $SVD $TAU $TGS $TRK BBCD BBCQ BBLD BBLU BBNK BFLD CALL CCRD CCSL CEMD CESD CESQ CESS CFWD CFWQ CHAD CHTD CMIO CMPD CMU3 CMUD CMUL CMUO CMUR CMUS CMUX CMXD CPRD CPRQ CSPD CSXD CTCD CTID DEDX DETS EBSD ELE3 ELES EVCD EVCL EVTX FEAD FEAQ FEMD FHAD FHXD FHXQ FMCD FMCE FMMD FMSD FMSX FMTD FMTE FMUD FMUL FMUO FMUR FMUS FMUX FRAD JET3 JETS L2JT L3HD LATD LRID LULD LUMD LUMI MET3 METS PEAD PEAQ PEMD PENX PESD PESL PESQ PESS PHAD PHWD PHWQ QMET QSVC QTOW QTRK QVTL SAPD SCLD SETA SLTE SLTM SLTT SV1D SVEL SVTD SVXD SVXE SVXK TAGZ TAUO TBMD TBMQ TCED TCMD TCMQ TCSD TDLF TEXD TFRD TL2D TL2Q TL3D TL3Q TMXD TNND TODD TOFD TPID TPXQ TTLD TUPD UPLD VTWD WHAD WHTD
Only the classes CMUO_Bank and LRID_Bank have been revised so far to include named accessor methods.
The program BNK_generate_headers will be extended to translate user command arguments (bank name and type string) to a boilerplate C++ header file describing such a bank. Experts in each bank's functionality will still want to add named accessor methods to these classes after they are produced, but much of the tedious work can at least be done automatically.
Plans for refining C++ header files in a multi-lingual environment will be discussed in a future version of this guide.
Plans for maintaining C++ header files in a multi-lingual environment will be discussed in a future version of this guide.
The goal of the Trybos Fortran API is to permit user modules written in Fortran to run as expected using Trybos, without YBOS software and with little or no modification. Hence, the Fortran API is just a subset of YBOS as described in the YBOS programmer's reference manual . The categories mentioned below are defined in that YBOS programmer's reference manual.
Subprograms in the following categories are to be implemented: Single Named Banks, Named Bank Internals, Sets of Banks, Subsets of Banks
Some subprograms in the following categories may be partially implemented: Work Banks, Initialization, Named Bank Validation, Utilities
Subprograms in the following categories will not be implemented: Alias Name Manipulations, Access to User Locations, Sequential I/O, Disk-based I/O, YBOS Array Free Space Monitoring
Comments on this page may be sent to Rob Kennedy