[Prev] [Next] [TOC]

Creating Custom ClearDDTS Reports

This chapter explains how the ClearDDTS report system is constructed and how it can be customized. Read this chapter if you want to modify the reports supplied with ClearDDTS or create new reports.

As shipped, ClearDDTS includes all of the source programs for the management reports so that these source programs can be modified to suit your needs. If you want to report additional information or produce custom reports, working examples are available to use as a starting point.

The following topics are covered:

Understanding how reports work

Each ClearDDTS interface expects reports and graphs to be produced in a particular format. For a report program to work, it must accept ClearDDTS parameters and produce output in the correct format. This section details those parameters and formats.

The report_conf file

One of the most important files in the ClearDDTS report mechanism is the ~ddts/class/<classname>/report_conf file. This file associates a report title with a report program and specific attributes. The ClearDDTS interfaces look here to see what reports are available in the formats appropriate for that interface.

To integrate a report into the xddts and webddts interfaces it must appear in the report_conf file. A copy of this file is shown below:

#                             report_conf
#
# This file drives the ClearDDTS reporting mechanism - note that the '#'
# character indicates a comment and all text from that character
# to the end of the line will be ignored.
#
# a line ending with a '\' character will be considered to continue
# on the next line. Each Report configuration line consists of three
# fields separated by the ':' character.  Each line looks like
#
#   attributes : Report Label Text : report command string
#
#                   R E P O R T  A T T R I B U T E S
#
#   -ascii  - the report's output is in ASCII format
#   -ps     - the report's output is in PostScript format
#   -troff  - the report's output is in Troff format
#   -notty  - the report should not be sent to a tty device.
#   -gif    - the report's output is in gif when run from the web
#   -html   - the report's output is in html when run from the web
#
#                   R E P O R T  L A B E L  T E X T
#
# The report label text is displayed in the list of available reports.
#
#               R E P O R T  C O M M A N D  S T R I N G
#
# The report command string is essentially a command similar to a string
# that would be passed to the system (3) call.  There are several
# variables that are understood and interpreted by the reporting 
mechanism.
# They are as follows.
#
#    $start    - start of date range for the reporting period
#    $end      - end of date  range for the reporting period
#    $projects - list of projects to be passed to report script
#    $states   - states argument to be passed to report script
#    $color    - Tells report script whether or not to generate
#                color output.
#
#
-ascii            : Table of Problems by Project by State.:\
                    dawk01.sh $start $end $states $projects # an appended 
comment
-ascii            : Table of Problems by Project by Severity.:\
                    dawk02.sh $start $end $states $projects
-ascii            : Table of Problems by Assigned Engineer by Severity.:\
                    dawk03.sh $start $end $states $projects
-ascii            : Table of Problems by Assigned Engineer by State.:\
                    dawk04.sh $start $end $states $projects
-ascii            : Table of Problems by Submitter by Severity.:\
                    dawk05.sh $start $end $states $projects
-ascii            : Table of Problem Arrival and Repair Rates.:\
                    dawk06a.sh  -b $start $end $states $projects
-ascii            : Three Line Summary of all Problems.:\
                    dawk07.sh $start $end $states $projects
-ascii            : General Problem Statistics.:\
                    dawk08.sh $start $end $states $projects
-ascii            : Report of Problems Submitted Over the Last Week.:\
                    wsubmit.sh $states $projects
-ascii            : Report of Problems Resolved Over the Last Week.:\
                    wresolve.sh $states $projects
-ascii            : Full Text Listing of Queried Problems.:\
                    wdump.sh
-ps -notty -gif   : Graph of Phase Metrics.:\
                    dgawk01.sh $color -b $start $end $states $projects
-troff            : Phase Containment Effectiveness Report.:\
                    dgawk02.sh -b $start $end $states $projects
-ps -notty -gif   : Phase Containment Effectiveness Bar Graph. :\
                    dgawk11.sh $color -b 1 $start $end $states $projects
-ps -notty -gif   : Total Faults Sourced Per Phase. :\
                    dgawk11.sh $color -b 2 $start $end $states $projects
-ps -notty -gif   : Graph of Problems by Resolution Type.:\
                    dgawk03.sh $color -b $start $end $states $projects
-ps -notty -gif   : Graph of Submitters, Assigned Engineers, and 
Resolvers.:\
                    dgawk05.sh $color -b $start $end $states $projects
-ps -notty -gif   : Graphs - Problems by Severity, Problems by State, 
Problem Arrival Rate.:\
                    dawk06.sh $color -b $start $end $states $projects
-ps -notty -gif   : Graph of Problems by Severity.:\
                    dgawk07.sh $color -b $start $end $states $projects
-ps -notty -gif   : Graph - Repair Time & Diff Between Est & Actual Fix 
Time.:\
                    dgawk08.sh $color -b $start $end $states $projects
-ps -notty -gif   : Graph of Overdue Problems. :\
                    dgawk09.sh $color -b $start $end $states $projects
-troff -notty     : One Line Meeting Summary of Selected Problems.:\
                    dgawk10.sh -b $start $end $states $projects

The active lines in the file are tuples separated by colons. The first parameter is an attribute that describes whether the output is ascii, postscript, gif, html, or troff output. The second parameter is used by webddts and xddts to produce the list of reports to select from. The last parameter is a string that is used to invoke the report.

There is a separate report_conf file for every class.

Report scripts

The default ClearDDTS reports consist of Bourne shell scripts that use awk to process the data. These files are located in ~ddts/bin. This directory contains many pairs of files named dawknn and dawknn.sh; for example, dawk09 and dawk09.sh. There are also pairs of dgawknn and dgawknn.sh scripts.

Each dawknn.sh or dgawknn.sh file is a shell script that runs the corresponding dawk/dgawk report script. Most of these shell scripts are similar except for the name of the awk script that they invoke.

How each interface uses these scripts to produce reports is detailed below.

xddts

To run a report, xddts executes the following process:

xddts accesses the report_conf file to obtain the list of reports to display on the Management Reports screen. It supports reports that are defined with the following attributes: -ps, -ascii, and -troff.
When a user selects a report or graph, xddts runs the shell script associated with that title using the values of the associated variables. For example, the Graph of Problems by Severity calls the dgawk07.sh script as defined in the report_conf file:

dgawk07.sh $color -b $start $end $states $projects

The dgawk07.sh script expands the variables and runs an associated report script, dgawk07, to produce the output.
In the case of a graph, an additional step is necessary to pass the information to the tallybug program, which in turn uses graphbug to produce a PostScript graph.

webddts

To run a report, webddts executes the following process:

webddts accesses the report_conf file to obtain the list of reports to display on the Generate Reports page. It supports reports that are defined with the following attributes: -ascii, -html, -gif, and -ps.
When a user selects a report or graph, the web reporting engine builds and runs a temporary shell script which:

expands a set of web-specific environment variables including:

calls the associated shell script (for example, dgawk07.sh)

The shell script does the following:

ignores the variables ($start, $end, $states, $projects) normally associated with the script in the report_conf file and processes the findbug query string produced by the webddts Query
runs awk (for reports) or tallybug (for graphs)

In the case of a graph, an additional step is necessary to pass the information from the tallybug program to either gifbug (to produce the graph in .gif format) or graphbug (to produce the graph in .ps format)

Creating Reports

You can write report scripts in any format that will accept ClearDDTS input variables and produce the expected output. As a start, you can edit the existing shell and awk scripts. For example, assume you want to write a ClearDDTS report to produce ASCII (.txt output) and call it report #22. To do this:

Go to the ~ddts/bin directory and copy the file dawk02.sh to dawk22.sh:

cd ~ddts/bin
cp dawk02.sh dawk22.sh

Change the dawk22.sh shell script to invoke your awk script called dawk22.
Create/modify the dawk22 script to produce the desired report. To do this, you need to know how to use awk and what defect records look like. Defect records are normal UNIX text files with lines of the form (see Appendix A for a complete description of the fields):

Keyword: value

A portion of a defect record file is shown below. Note that these files always begin with the "Start: bug-ID" field and end with the "End: bug-ID" field.

Start: CMMaa000135
Project: DDT.bugs
Engineer: mike
Submitted-on: 880606
Headline: The clinit(3) routine dumps core in diagnostic mode
Severity: 2
Status: N
        .
        .
        .
End: CMMaa000135

/^Severity:/ { 
              severity = $2 
              next 
             } 

/^Status:/   { 
              status = $2 
              next 
              } 

/^Headline/   { 
               for(i = 2; i <= NF; ++i) 
               headline = headline $i " " 
               next 
              } 
              . 
              . 
              . 
              { 
               printf "\nBug Number = %s\n",xstart. . . 
               printf "St=%s, Sv=%s, %-65.65s\nModule: %s, . . . 
              }

GIF reports

For webddts to produce graphs in gif format the report must pay attention to the DDTS_GRAPH_FORMAT environment variable. If it is set to "gif", the report must call the program gifbug. The gifbug program operates much the same as graphbug (it parses tallybug output), but it produces gif instead of PostScript output.

Since multiple gifs can be generated, the gifs need to be created in a certain way so that webddts knows how and where to view them:

if [ "$DDTS_GRAPH_FORMAT" = "gif" ]; then 
  gifbug < tmp/dg5tmp1.$$ >$home/www/$DDTS_GRAPH_BASE.01.gif 
2>$home/www/$DDTS_GRAPH_BASE.01.err.txt 
  gifbug < tmp/dg5tmp2.$$ >$home/www/$DDTS_GRAPH_BASE.02.gif 
2>$home/www/$DDTS_GRAPH_BASE.02.err.txt 
  gifbug < tmp/dg5tmp3.$$ >$home/www/$DDTS_GRAPH_BASE.03.gif 
2>$home/www/$DDTS_GRAPH_BASE.03.err.txt 
  gifbug < tmp/dg5tmp4.$$ >$home/www/$DDTS_GRAPH_BASE.04.gif 
2>$home/www/$DDTS_GRAPH_BASE.04.err.txt 
  echo "$Title1" > $home/www/$DDTS_GRAPH_BASE.00.desc 
  echo "$Title2" >> $home/www/$DDTS_GRAPH_BASE.00.desc 
  echo "$Title3" >> $home/www/$DDTS_GRAPH_BASE.00.desc 
  echo "$Title4" >> $home/www/$DDTS_GRAPH_BASE.00.desc 
  rm -f tmp/dg5tmp1.$$ tmp/dg5tmp2.$$ tmp/dg5tmp3.$$ tmp/dg5tmp4.$$ 
  exit 0 
else 
  graphbug $colorflg < tmp/dg5tmp1.$$ > $home/tmp/ps.dg5awk1.$$ 
  graphbug $colorflg < tmp/dg5tmp2.$$ > $home/tmp/ps.dg5awk2.$$ 
  graphbug $colorflg < tmp/dg5tmp3.$$ > $home/tmp/ps.dg5awk3.$$ 
  graphbug $colorflg < tmp/dg5tmp4.$$ > $home/tmp/ps.dg5awk4.$$ 
fi

This code snippet shows that gifbug is called in much the same way as graphbug, but the output filename has changed.

In the case of multiple gifs, each gif should be directed into its own sequentially numbered file. Furthermore, the description file contains one title per line. Webddts associates the first line of this file with "01.gif", the second line with "02.gif" and so on. When the report is complete, webddts lists them something like:

_Page 01_ Problems by state 
_Page 02_ Problems by severity 
_Page 03_ Problems by engineer 
_Page 04_ Problems by phase

This clarifies what each gif will display when you click on the link. Be sure that the description file is numbered "00" so that it sorts before the other gif files, otherwise it will not get picked up.

You can customize the gif display sizes available in the webddts interface and set the default value (640x480 as shipped). To do this edit the ~ddts/class/<class>/web_conf file. You can also customize the number of x-axis and y-axis labels and ticks through the web_conf file.

HTML reports

For html reports, the report itself must generate the appropriate html tags so that the web browser can interpret it properly. Three color environment variables are set that can be used in HTML reports:

        DDTS_THCOLOR # table header color, for html reports 
        DDTS_BGCOLOR # page background color, for html reports 
        DDTS_TEXTCOLOR # text color, for html reports

Including these environment variables keeps the report visually continuous with the rest of the web interface.

You can also provide links to defects in the report by building URLs such as:

  <a href=$DDTS_WEBPROG?REMOTE_USER=$DDTS_REMOTE_USER&NextForm=DumpBug&bug_id=$bug_id>

where $bug_id is a defect identifier that you have calculated. This URL displays only the defect record, not the full frames mode of webddts. To get the full frames mode, specify id=$bug_id instead of bug_id=$bug_id.

If there is a whole list of defects, you can display them in a new window by specifying a target in the URL.

Integrating a report into the xddts and webddts interfaces

To integrate a new report into the xddts and webddts interfaces, you need to look at the ~ddts/class/<class>/report_conf file. This is where the interfaces look for the list of reports to display.

Note that the reports being invoked in the default report_conf file are the dawknn.sh shell scripts. Reports you add can also be awk scripts or any other type you have created that will produce the expected output.

To add your dawk22.sh report from the previous example to the report_conf file, you would add something like the following:

-ascii:   This is the foo report:   dawk22.sh $states $projects

[Prev] [Next] [TOC]