6. Data Display (Data Consumers)

6.1.1. Tabular Suite Summary View

The first item in the CURRENT DATA menu executes status.jsp with default.xsl to create a summary table of suite results. This JSP takes comma-delimited lists in its suiteNames and optional resourceIds parameters that specify the suites and resources to be shown. The default installation invokes http://localhost:8080/inca/jsp/status.jsp?suiteNames=sampleSuite&resourceIds=defaultGrid, producing an image similar to this one:

6.1.2. Map View using Google Map API

6.2.1. Pass/Fail Graphs and Error Messages

The first item in the REPORTS menu executes report.jsp, which uses CeWolf/JfreeChart, to generate historical summary reports with pass/fail status and error information.

The graphed series can be customized using the page described in Section 6.4.2. Each "graph" folder represents a separate graph on the page and the series under it will appear in the graph. Series should be entered in the format "nickname,resource,label" and correspond to the names and resources of committed series. Each graph has its own title. The height and width of all the graphs can also be customized as well as the page description and title.

The default consumer groups different combinations of sampleSuite series and displays the past week of results. Each set of series has a title (e.g., Software Deployment Tests) for its grouping and the series nickname, resource name, and report label for each series in the group.

6.2.2. Metric Reports

Like above, the first item in the REPORTS menu executes report.jsp, which uses CeWolf/JfreeChart, to also generate historical summary reports graphing one or more metrics.

The graphed series can be customized using the page described in Section 6.4.2. Each "graph" folder represents a separate graph on the page and the series under it will appear in the graph. Series should be entered in the format "nickname,resource,label" and correspond to the names and resources of committed series. The default consumer graphs a single metric called 'bandwidth' for the series "wget_performance,localResource,wget test". Additional metrics can be specified as a comma separated list, e.g., "EP-STREAM_Triad_GB_s,G-STREAM_Triad_GB_s,Wall_Mins".

There are three different chart types that can be specified as defined below. By default, the chart type will be 'metric'.

• metric: one graph containing all series will be printed per metric

• series: one graph containing all metrics will be printed per series

• single: all series and metrics will display on a single graph

Below is the graph configuration for two series, one connecting to the alamo resource and the other to the foxtrot resource. Each series collects two metrics - ping and ssh. There are two graphs that are identically configured except that the first one uses the "single" chart type so that all series and metrics are graphed together, and the second uses the "metric" chart type so that the ssh and ping metrics are graphed separately.

The graphs created by the configuration above are shown below. The first graph uses the chart type "single" so we see four lines, one for each series and one for each metric:

The second configuration uses the chart type "metric" so there are two graphs, one for each metric. Each graph has two lines, one for each series:

If configuration used the chart type "series" there would be two graphs, one for each series. Each graph would have two lines, one for each metric:

6.2.3. Average Pass Rate by Resource/Suite

The second item in the REPORTS menu executes summary.jsp, which uses CeWolf/JfreeChart and stylesheets seriesAverages.xsl and periodAverages.xsl to show the average series pass rate by resource and suite for the past week. Each bar label shows the value of the average series pass rate for the last week and the difference in percentage from the previous week. The color is green if the average pass rate is better than the previous week, red if the average pass rate is worse, and gray if there was no change. Individual bars can be clicked on to show the percentages broken down further for each individual resource or suite.

Click on any bar to view a more detailed report for that resource or suite. For example, clicking on the bar for the 'app-support.teragrid.org-4.0.0' suite above shows the average pass rates for the 'app-support.teragrid.org-4.0.0' suite for each resource (figure below).

6.2.4. Series Error Summary

The third item in the REPORTS menu executes status.jsp with seriesSummary.xsl to summarize test series errors by time period. Each time period includes the number of errors for the series during the time period, the number of unique or distinct errors during the period, and the percentage of the total results that passed during the period.

The time periods and titles (i.e., PAST 4 WEEKS) can be customized using the page described in Section 6.4.2.

The change between the total number of errors in the most recent period and the total number of errors in the period before it is also given. If the number of errors in the most recent period is greater than the number of errors in the previous period (+), the number appears in red. If the number of errors in the most recent period is less than the number in the previous period (-), the number is green.

6.2.5. Average Pass History

The fourth and fifth items in the REPORTS menu execute summaryHistory.jsp, which uses CeWolf/JfreeChart and stylesheets seriesAverages.xsl and periodAverages.xsl to show the average series pass rate over time grouped by resource and suite. I.e., When grouped by suite, a history graph summarizing the average series pass rate is displayed for each suite. By default, multiple lines are used to show the summary history on each resource. To view the total summary history percentages for all resources, click the total checkbox in the top right corner and click the 'Filter' button. Histories for either specific resources or suites can also be displayed thru the form in the top right corner. The amount of history shown is 4 weeks by default but can be modified by editing the 'incaQueryStatus' query parameters described in Section 6.3.3.

6.3.1. Create Graph

The first item of the QUERY menu executes status.jsp with graph.xsl to create a form that allows you to graph the history of more than one series. The form allows you select the set of series of interest via a set of checkboxes.

After selecting the "vtk-nvgl_version" series and clicking on the "graph" button in the form above, a graph page similar to the one below is shown with an XY plot of the pass/fail status of the selected series, a series pass/fail summary table, a bar graph of error message frequency (if the selected series have errors), a summary table of error messages, and a form to customize the graph further.

To query for all results, clear the "start date" field in the "customize graph" section and click "re-graph". To retrieve graphs more quickly, data point mouseover text and links to report details are turned off for the pass/fail graph. The data points can be made interactive by choosing "show mouseovers/hyperlinks" in the "customize graph" section and clicking on "re-graph". Clicking on interactive data points will lead to report details pages, mousing over them will show collection times and any error messages.

Multiple series can also be graphed together as in the image below. A table under each graph summarizes results for each series.

Graph pages for individual series are also linked from the report details pages.

6.3.2. Create Stored Query

The second item in the QUERY menu executes status.jsp with create-query.xsl to provide a form for creating a cached query. Unless password protection has been disabled as described in Section 6.15, selecting this item causes the consumer to prompt for an id and password. The consumer can cache frequently executed depot queries in order to improve data display speed. For example, the latest results for a suite can be cached in order to quickly display a suite status page. By default the latest results for the sampleSuite are cached every 2 minutes. Results are also cached each day for historical reports.

Cached queries can be used to display custom tables of latest results, using the queryNames parameter to status.jsp For example, the image below shows the creation of a cached query, named gccTests, for two series in the default installation.

Once this query has been stored and executed, the table shown below can be viewed via http://localhost:8080/inca/jsp/status.jsp?queryNames=gccTests&resourceIds=defaultGrid

6.3.3. Manage Stored Queries

The third item in the QUERY menu executes query.jsp to allow management of cached queries. Unless password protection has been disabled as described in Section 6.15, selecting this item causes the consumer to prompt for an id and password. Click on one of the pre-defined queries to see its parameters. Each query has a name, execution schedule, depot command it executes and the parameters for the depot command executed.

By default, the latest instances of all suites are cached every 2 minutes by predefined queries named incaQueryLatest+agentUri_suiteName. These cached queries are used for the CURRENT DATA menu views described in Section 6.1. Another default query is "incaQueryStatus" which retrieves a summary of the number of successes and failures for each series for the past 4 weeks. This cached query is used for the several of the reports described in Section 6.2. To display 10 weeks of history instead, modify "28" days to "70" and click "Change". Note that the consumer will automatically re-create any of these default queries that are deleted.

6.4.1. List Running Reporters

The "Running Reporters" page displays running reporters series by suite. Each series lists its name, frequency of execution, whether email notification is enabled, the reporter script used for the series, and a description of the reporter script. Note that in this view, and in the more detailed running reporters view, the reporter names link to a CGI script that will not work for repositories that are not web accessible (like the local default repository). Make your repository web accessible and add/commit the repository URL beginning with "http" in order to activate the reporter links.

The detailed "Running Reporter Series" page below lists the running reporter series and the suite each belongs to, the machine each is scheduled to execute on, whether email notification is configured, and the recipient ("Target") of any notification.

6.4.2. View/Change Configuration

Here Inca administrators can view and change the global configuration of status pages. Determine whether to allow run nows or the execution of tests via the detailed status pages (see Section 6.5.1) and configure the knowledge base, google map, series summary and status report status pages.

6.5.1. Run Nows

If the "Allow run nows from web pages" option is checked (see Section 6.4.2), a "Run Now" button will appear under the "Command used to execute the reporter" heading. This option is useful for system administrators who have fixed an issue and want to see the updated result or for those who want to verify that a problem has been resolved.

Clicking on the "Run Now" button triggers reporter execution on the local machine. Depending on the reporter execution time and page cache frequency (see Section 6.3.3), results may take a few minutes to propagate to the Inca web pages.

Since authentication is required based on the authentication setup (see Section 6.15), a box will popup such as follows:

Press 'Continue' and the security login window will display such as follows:

Use the username and password specified in $INCA_DIST/etc/realm.properties. By default, the username and password are 'inca'. After the consumer submits the request, you will see a confirmation such as below:

6.5.2. Knowledge Base

The Inca default knowledge base is designed to collect problem resolution information for tests. System administrators may use solutions stored in the knowledge base when debugging issues on their resources. The knowledge base can be configured using the admin page described in Section 6.4.2 with the options described after the configuration image below:

• enable:

  If the "enable" value is set to "true" then a "search knowledge base" button and an "add to knowledge base" button will appear under the "Result" heading of the reporter details status pages. If "enable" is set to "false" then the knowledge base buttons will be hidden and inaccessible.

• searchString:

  The URL that will be visited after clicking on the "search knowledge base" button. The macros "@nickname@", "@reporter@" and "@error@" will be replaced by the actual series nickname (e.g. gcc_version), reporter name (e.g. cluster.compiler.gcc.version), and error message for each test. The "+" character will be replaced by "&".

• submitString:

  The URL that will be visited after clicking on the "add to knowledge base" button. The macros "@nickname@", "@reporter@" and "@error@" will be replaced by the actual series nickname (e.g. gcc_version), reporter name (e.g. cluster.compiler.gcc.version), and error message for each test. The "+" character will be replaced by "&".

• submitEmailNotification:

  The email address to send new, changed, and removed knowledge base articles to. By default no email notifications are sent and the "submitEmailNotifications" value is set to "none". To add email notifications, change this value to an email address or a list of email addresses separated by commas (e.g. "email1@loc.edu, email2@loc.edu").

Once the knowledge base is enabled, two buttons to search and add to the knowledge base will appear on the report details pages:

Clicking on the "search knowledge base" button will lead to the URL set as the "searchString" value. By default, this returns all of the knowledge base articles related to the series:

Clicking on the "add to knowledge base" button will lead to the URL set as the "submitString" value. By default, this is a form for entering article information. Since authentication is required for this form based on the authentication setup (see Section 6.15), a box will popup such as follows:

Press 'Continue' and the security login window will display such as follows:

Use the username and password specified in $INCA_DIST/etc/realm.properties. By default, the username and password are 'inca'. After the consumer submits the request, a form will appear to complete like the one below:

6.9.1. Colors and Fonts

Properties such as colors and fonts are controlled by the default CSS (Cascading Style Sheet) file $INCA_DIST/webapps/inca/css/inca.css. The drop down nav bar in the header is controlled by $INCA_DIST/webapps/inca/css/nav.css. You can edit these files to customize the consumer display. For example, to change the color in the reporter detail pages header bar, edit inca.css and change lines 111-112 to:

.header {
          background-color: #D07651;

For general information, visit the [ CSS tutorial ].

6.9.2. HTML Layout

To modify the default HTML layout, edit the XSL stylesheet that is being applied to the JSP or create a new stylesheet in $INCA_DIST/webapps/inca/xsl and pass it in the JSP "xsl" parameter. The default JSP and XSL files are described in Section 6.7.

For general information about editing stylesheets, visit the [ XSL tutorial ].

188  <xsl:variable name="cellText">
            189    <xsl:choose>
            190      <xsl:when test="string($instance)=''">
            191        <xsl:value-of select="''" />
            192      </xsl:when>
            193      <xsl:when test="string($result/body)!=''
            194             and string($result/errorMessage)=''
            195             and ($comparitor='Success' or count($comparitor)=0)">
            196               passed: 
            197        <!-- get yyyy-mm-dd from gmt timestamp -->
            198        <xsl:value-of select="substring($result/gmt, 1, 10)" /> 
            199        <!-- get HH:MM from gmt timestamp -->
            200        <xsl:value-of select="substring($result/gmt, 12, 5)" />
            201      </xsl:when>
            202      <xsl:otherwise>
            203               error:
            204        <xsl:value-of select="substring($result/errorMessage, 1, 30)" />
            205      </xsl:otherwise>
            206    </xsl:choose>
            207  </xsl:variable>
            208  <xsl:choose>
            209    <xsl:when test="$exit!=''">
            210      <td class="{$exit}">
            211        <a href="{$href}"><xsl:value-of select="$cellText"/></a>
            212        <xsl:if test="$url[matches(., 'markOld')]">

6.9.3. Custom Page Header

The default page header is generated by the inclusion of the header.xsl file in other xsl files. The header is a navigation bar with drop down links to a set of default status pages:

The header's navigation bar is an HTML unordered list that is formatted with the nav.css stylesheet. To add or remove links in the navigation bar, open header.xsl and change the appropriate link. For example, to show a table of suite results for a new suite called "newSuite" on a new resource called "newResource":

57  <li><h2>Current Data</h2>
58    <ul>
59      <li>
60        <a href="'status.jsp?xsl=default.xsl&amp;suiteNames=newSuite&amp;resourceIds=newResource'">
61           table of newSuite results</a>
62      </li>

Any other link can be added or removed as a list element. The stylesheet supports additional levels of nested list links. Note that the top level list elements are displayed from bottom to top (i.e., CURRENT DATA is listed last in header.xsl so it is displayed the furthest left in the top level navigation bar).

Another example of customizing the HTML header would be to add a call to a custom header stylesheet in the "printBodyTitle" template. Edit inca-common.xsl:

13  <xsl:include href="custom-header.xsl"/>
          ...
          24  <xsl:template name="printBodyTitle">
          25    <xsl:param name="title"/>
          26    <xsl:call-template name="custom-header"/>
          27    <xsl:variable name="datenow" select="date:new()" />
          28    <xsl:variable name="dateformat" select="sdf:new('MM-dd-yyyy hh:mm a (z)')"/>
          29    <table width="100%" border="0">
          30      <tr align="left">

Create a $INCA_DIST/webapps/xsl/custom-header.xsl like:

<?xml version="1.0" encoding="UTF-8"?>

          <!-- ================================================ -->
          <!-- Prints out custom header for Inca status pages   -->
          <!-- ================================================ -->
          <xsl:stylesheet version="1.0"
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
          xmlns="http://www.w3.org/1999/xhtml">

          <xsl:template name="custom-header">
          <table class="header" width="100%">
          <tr>
          <td bgcolor="#003366">
          <img class="logo" src="img/header.jpg"/>
          </td>
          </tr>
          </table>
          <table class="menu" width="100%">
          ... custom navigation ...
          </table>
          </xsl:template>

          </xsl:stylesheet>

The resulting page would look something like: