PBS XML Accounting Toolkit
Rodney Mach < r m a c h at umich edu >
Intro
The PBS accounting toolkit takes standard OpenPBS or PBSPro batch scheduler
accounting data for Linux and converts it to an XML representation for easier
transformation/rendering to other formats.
Also includes java code to do these
transformations as well as supporting XSD, and XSL files for validation and
transformation to other formats such as NPACI joblog format, and
the Global Grid Forums UR-WG UsageRecord accounting format (see here)
If you use this package, you can create slick accounting reports
using the DarkSlide reporting
engine to create fancy PDF's to impress your boss!
Latest News
NMI will be including the accounting toolkit in release 5!
Support for special Torque fields will be forthcoming in the next minor release!
Article appears on PBS Accounting toolkit in the October issue
of sysadmin magazine!!
Latest version 1.4.3 of Accounting Toolkit supports full draft
of the UR-WG UsageRecord accounting format from the GGF for
both conversion AND the darkslide reporting engine! Also
in addition to PDF reports, Excel reports can also now be
generated by the darkslide reporting engine
Requirements
To use the package you will need J2SDK 1.4 or higher from
Sun http://java.sun.com/ If Java is not
available for your platform, you can generate the XML according to the
DTD/Schema in the XML directory in your language of choice, and use the
corresponding XSL files to perform any translations.
Example
An example output can be found Here
An example PDF using the corresponding darkslide report engine is here
Usage
Converting accounting data to XML
In
/usr/spool/PBS/server_priv/accounting/ directory is the accounting files for OpenPBS.
For PBSPro it is /var/spool/PBS/server_priv/accounting/.
To convert an accounting file to it's XML representation type (at Unix prompt ) : pbstoxml < file.dtd|file.xsd> account_filename output.xml
For example, to convert the accounting file named 20021217 and save it to
20021217.xml you would type cd /usr/local/pbsaccounting/bin
pbstoxml /usr/spool/PBS/server_priv/accounting/20021217 20021217.xml
This does not assign a Schema to the document so that you can
validate it.
To assign a W3C Schema, type
pbstoxml ../xml/linux_joblog.xsd 20021217 20021217.xml
The XSD file is included in the distro.
Converting XML to NPACI joblog format
Once you have converted the
accounting data to XML, you can convert the XML to npaci joblog format by typing
pbstojoblog file.xml joblog.txt
For example, to convert the previous 20021217.xml file to an NPACI joblog
formatted file you would type pbstojoblog 20021217.xml joblog.txt
Converting XML to Global Grid Forums UR-WG UsageRecord format
Once you have converted the accounting data to XML, you can convert
it to the UsageRecord XML format as follows
pbstogrid file.xml grid.xml hostname ppn
For example, to convert the previous 20021217.xml file to a UR-WG
XML file you would type pbstogrid 20021217.xml grid.xml
Note that this supports the final draft version UsageRecord schema
as of October 2003.
Limitations
Record Types
Currently all major OpenPBS records other than CheckPoint records are supported (if you need
this send me an example record I will add it in) and I believe most PbsPro records.
If you discover unknown record types (a warning is issued on conversion) please
send them to me. Any types not supported are because I don't have examples
of them
Platforms
I have tested on Linux only.It should work on other platforms,
but since they will have different resources etc the code will need to change
to support them. If you need support on your platform feel free to contribute
any code. I will be adding support for the Origin and supporting schema in a future version.
Resource List
The list of resources for resource list is scheduler
specific. I included ones that we use at our site with the default scheduler, if
you have other resources you want to add it should be straightforward to support
these.
License
The source code is released under the LGPL. Click here for license information
Requesting features, Reporting bugs
Submit any bugs or requests here here.
Developer Document Listing
- Full API
documentation
- Linux
Joblog Schema documentation
- Unit
testing
- Unit testing coverage report for release v1.4.2
- Building
Installing via RPM
Go to the files section here here and download the RPM for pbsaccounting. If you want to
use the darkslide package with it, you will ALSO need to download the darkslide
RPM. The current version of pbsaccounting is pbsaccounting-1.4.2-1.i386.rpm,
and current release of darkslide is darkslide-1.10-1.i386.rpm
Downloading the code
http://sourceforge.net/projects/pbsaccounting/
has the source code available from CVS where you can build your own distro,
includes all the documentation and unit testing. Here is how to get it from CVS
(copied from sourceforge docs)
% export CVS_RSH=ssh
% cvs -d:pserver:anonymous@cvs.pbsaccounting.sourceforge.net:/cvsroot/pbsaccounting login
% cvs -d:pserver:anonymous@cvs.pbsaccounting.sourceforge.net:/cvsroot/pbsaccounting checkout pbsaccounting
I have noticed that the anonymous checkout is old however, sourceforge
doesn't always have the latest CVS checkin available for whatever reason
via anonymous CVS