There are a couple basic steps which are detailed below

Have all your pbs accounting data converted to XML. See the pbsaccounting package for the RPM and instructions
You set up Xindice XML database, and load that XML data into it
You generate your report

Install Xindice

DarkSlide uses the native XML database Xindice to store the accounting data. Here is the procedure (also documented in the Xindice admin guide) to setting up the database to prepare for storing the accounting data.

Download Xindice xml-xindice-1.0.tar.gz from http://xml.apache.org/xindice

Install Xindice and start it up

% gunzip xml-xindice-1.0.tar.gz
% tar xf xml-xindice-1.0.tar
% mv xml-xindice-1.0 /usr/local/
% nohup /usr/local/xml-xindice-1.0/start &

Now if everything went well, you should get a message saying "Server Running"

Set important environment variables

These must be set in your shell, this assumes BASH

 
% export PATH=$PATH:/usr/local/xml-xindice-1.0/bin
% export XINDICE_HOME=/usr/local/xml-xindice-1.0

Create the collection.
You now must create a "collection" to store your XML accounting data in. By convention I just use the name of the host that the server is collecting data for, in this case the server is "morpheus". (remember what you use here, as later you will need to edit the config file to reflect the name of the collection you chose )
```
% xindiceadmin add_collection -c /db/ -n morpheus
Created : /db//morpheus
```

Install Darkslide

To install darkslide, download the RPM from http://sourceforge.net/projects/pbsaccounting/ and install as follows

% rpm -i darkslide-1.10-1.i386.rpm

Then you will need to edit the config QueryEngineConfig.xml file in /usr/local/darkslide-1.10/etc/. The most important parameter is to make sure the <collection> tag has the same collection name you created back in the "Create the collection" step. The default is "morpheus" which is the same name used in these examples, it is arbitrary and I generally use the hostname as mentioned previously.

You may also want to edit the title tag <title> that modifies the title of the report. You should not have to edit any other info on this file.

If you have any problems you may want to lower the debugging tag <debugging> from SEVERE to FINEST to get copious amounts of debugging information.

Load your XML accounting data

Assumingly you already a directory of XML accounting data produced by the PBS accounting toolkit at http://pbsaccounting.sourceforge.net. You can use this directory to do an initial "bulk load" of them into Xindice. This is a one time step.

Assuming all my XML data from a Pbs installation is in /usr/spool/PBS/server_priv/accounting/xml/

% /usr/local/darkslide-1.10/bin/bulkloader --directory=/usr/spool/PBS/server_priv/accounting/xml/ 
Document /usr/spool/PBS/server_priv/accounting/xml/20030331 inserted
Document /usr/spool/PBS/server_priv/accounting/xml/20030401 inserted
Document /usr/spool/PBS/server_priv/accounting/xml/20030402 inserted
Document /usr/spool/PBS/server_priv/accounting/xml/20030403 inserted
Document /usr/spool/PBS/server_priv/accounting/xml/20030404 inserted
Document /usr/spool/PBS/server_priv/accounting/xml/20030405 inserted

You can verify that the database contains the documents by using an Xindice command to query the files in the collection. In this case to query the morpheus collection the command is as follows

% xindiceadmin ld -c /db/morpheus

Generate the PDF report

To generate a report from 4-10-2003 through 04-29-2003 and save report in a file called /tmp/test.pdf type the following command:

 /usr/local/darkslide-1.10/bin/reportgenerator --startdate=04-10-2003 --enddate=04-29-2003 --filename=/tmp/test.pdf

To generate a report for a single day, like the 29th of April, the command would be

 /usr/local/darkslide-1.10/bin/reportgenerator --startdate=04-29-2003 --enddate=04-30-2003 --filename=/tmp/test.pdf

Creating Excel reports

To create excel reports simply give an ".xls" extension to your output file. For example

/usr/local/darkslide-1.10/bin/reportgenerator --startdate=04-10-2003 --enddate=04-29-2003 --filename=/tmp/test.xls

Using URWG XML accounting data instead of PBS XML accounting data

If you populate Xindice with XML accounting data corresponding to the global grid forums URWG XML Schema instead of PBS XML accounting data, you can still use darkslide to generate reports! You must change one line in the configuration file as follows

Open up QueryEngineConfig.xml config file
edit the module tag to tell it to use a different module for parsing XML data. Change <module> to read:
```
umich.mgrid.queryengine.UrwgLookupService
```

Note you can't mix URWG XML accounting data with PBS XML accounting data.

Daily updates

To load your daily XML accounting data into Xindice, simply create a daily cron job that loads the XML document into Xindice using the xindice commands here:

xindice add_document -c /db/morpheus -f /usr/spool/PBS/server_priv/accounting/xml/filename -n filename

where the filename is the name of the accounting file, like 20030418

Advanced querying

By loading the accounting data into a native XML database like Xindice, you get the power of doing any type of valid XPATH query against your data. For example, if you want to know all the days that the user "rmach" had completed jobs, you can use the xindice xpath command with the appropriate XPATH query as follows:

% xindice xpath -c /db/morpheus -q "//pbs_jobfile/execution_record[user='rmach']/date/day"

To use this mechanism you will need

To know XPATH. A good tutorial is here
To know the XML Schema for the accounting data, see the linux_joblog.xsd included in the pbs accounting toolkit distro.

Developer Document Listing

Downloading the code

http://sourceforge.net/projects/pbsaccounting/ has the source code available from CVS where you can build your own distro, includes all the documentation and unit testing. Here is how to get it from CVS (copied from sourceforge docs)

% export CVS_RSH=ssh
% cvs -d:pserver:anonymous@cvs.pbsaccounting.sourceforge.net:/cvsroot/pbsaccounting login 
% cvs -d:pserver:anonymous@cvs.pbsaccounting.sourceforge.net:/cvsroot/pbsaccounting checkout pbsaccounting

Darkslide. The PBS Report Generator

Overview

Example PDF Output

Architecture

Requirements

Installation