Tips on Building Customized Reports against Netcool Event Data

This section provides a set of guidelines you can follow when designing reports. These are all based on the reporter data source populated by the Netcool Reporter gateway.

Tip 1: Structure and Modify Originating Data to Support Reporting

Netcool Reporter locates, queries, and presents Netcool/OMNIbus event data. This means that any real-time data stored or displayed in the ObjectServer can be reported on by users of Netcool Reporter. To preserve the high-speed, real-time performance on the ObjectServer, Netcool Reporter passes the ObjectServer event data to a relational database (through the Netcool Reporter gateway) before querying that data through the reporting application. The data reaches the relational database in the same format in which it leaves the ObjectServer. The alerts.status, alerts.journal, and alerts.details tables are passed directly across the gateway, and the REPORTER_STATUS table and Netcool Reporter gateway can also be edited to pass across any additional, custom fields from your ObjectServer alerts.status table. Other key tables for reporting, such as members, classes, conversions, groups, names, and members, are passed across from the ObjectServer to the relational database through a special gateway utility.

Because the ObjectServer is the primary data source for queries executed by Netcool Reporter this means that the data you want to report on must first exist and be properly formatted within the ObjectServer. For example, if you want to report on faults by geographical location, the Location field in the ObjectServer must be populated accurately and consistently with geographic data. To take this a step further, if you want to report on faults by business customer, you must first create an additional field in your ObjectServer status table called Customers, then ensure that this field is populated accurately and consistently with customer data.

Another important consideration is the format of the data strings, dates, or numbers contained within the ObjectServer fields. The Netcool Reporter gateway sends the data across in the same format in which it is stored and displayed in the ObjectServer. In other words, node names, location names, user IDs, and so on, are transferred across with whatever plain English, code-based, or abbreviated strings operators use to analyze event data in the ObjectServer. While this may suit a small team of internal operators, it may not be appropriate for external audiences who want to view reports on network event data. One option is to modify the way data is being stored in the ObjectServer to make it more report-friendly. Another, perhaps more easily implemented, solution is to ensure that ObjectServer conversion tables are populated to the fullest extent to provide plain-language cross-references to the codes, strings, and abbreviations stored in the ObjectServer.

Finally, date-based and quantitative reporting measures depend on the accuracy of date strings, time stamps, numerical entries, and so on, into the ObjectServer data tables. To ensure accurate reporting, it is necessary to pay attention to a number of variables, possibly including:

•   Time and date settings on the ObjectServer machine.

•   Reconciling time and date settings across multiple machines.

•   Ensuring that acknowledgments are tracked and recorded accurately and promptly.

•   Ensuring that changes in ownership are tracked and recorded accurately and promptly.

Tip 2: Manage the Size and Length of Queries

When building customized reports, you are advised to follow some basic guidelines for building, testing, and distributing reports.

Building Reports

The first step in designing a report is to define, as narrowly as possible, the desired results set. Any extraneous data that does not support the underlying query must be excluded from the query design. For example, if only the past week's data is relevant, then a time-based condition or parameter must be designed into the query. If only three nodes are relevant, then the report must be designed with a constraint or a parameter that excludes all irrelevant nodes. Consider also the number of rows you want to return, as well as ways to summarize or condense raw data into more compact, meaningful reports. For example, if you are interested in finding out about events that are taking too long to acknowledge, define a threshold above which you want to scrutinize the data, for example, more than 60 seconds, instead of asking for a report that returns the acknowledge period of every single event.

One way that queries can be speeded up is through the use of indexing on database tables. Setting an index on a specific field can enable a query to run more quickly and efficiently against that field. For example, if you frequently query your data based on a grouping by Node, then it is a good idea to index the Node field in the REPORTER_STATUS table in your relational database. Indexing tends to be a database administrator task, so contact your on-site DBA resources for support with indexing.

Another way queries can be speeded up is through partitioning and archiving of irrelevant data. For example, if your active data querying was always on data generated within the last month-long period, then it is a good idea to partition off, or archive, data that is more than one month old. Once this data is moved from the active database, then all queries against the active data no longer have to check against the older records. Partitioning and archiving tend to be database administrator tasks, so contact your on-site DBA resources for support with partitioning or indexing.

A final mechanism that can be helpful in speeding up both inserts into and queries against the relational database is filtering or suppressing unnecessary data before it enters the relational database. The Netcool Reporter gateway includes filtering mechanisms that can keep extraneous data from being forwarded to the relational database. Your Netcool administrator can assist with modifications to the gateway configuration file.

Testing Reports

While you are designing customized reports, you may find that you need to go through a few iterations before you obtain the desired results and format. The Report Designer includes the ability to preview report definitions before saving them or turning them into static snaphots. While you are in the early stages of building a customized report, you may want to limit your preview reports by setting a delimiter on the number of data rows analyzed and returned. This can be done through the Report Properties button (on the Report tab or Chart tab), which includes a setting for number of rows returned. When you are satisfied with the report definition, you can remove the delimiter and query the entire active database.

Bear in mind that previewed reports are generated as HTML files that go into the websqlbr/temp directory. Over time, these files build up if they are not cleared from the temp directory. The application includes a cron job to clear this file directly, but you may also wish to empty the file manually if you are running an extensive number of previewed reports. Note that when you build and preview drill-down reports, you are actually generating a temporary file for every single drill-down page. This causes an even faster buildup of temporary files.

Distributing Reports

After you have generated a satisfactory query, you may wish to distribute access to other Netcool Reporter users. You have many options for distributing reports, each of which has slightly different implications for the relational database and the reporting server.

One option is to create and distribute a report definition. When you do this, you are giving every recipient the ability to query the active database. Opening a report definition means engaging in a session directly with the database so that the most up-to-date event data may be included in the report. This may be ideal for ensuring that end users receive the most current and accurate data. This also allows users to manipulate and interact with the returned report data. However, if several people are running and viewing the query simultaneously, this may put a considerable load on the relational database.

A second option is to distribute a report snapshot. When you do this, you are distributing a static, HTML file containing information that was generated at a specific point in time. This may be ideal when distributing the same information to large numbers of people, when the report needs to refer to a fixed time period (for example, the month of March), and when you want to constrain access to the active database. However, static snapshots cannot be manipulated by end users in any way, nor do they allow report users to place any parameters on the information they want to view. This places more of the burden on you, as the report designer, to generate exactly the type of report end users want to view.

A third option is to generate reports in other formats, for example, PDF, PNG, or Excel 97. When you as the report designer specify a report to output in PDF or PNG, you are creating a file that goes into your infospc/packages/printing folder.

Tip 3: Set Up a Logical User Environment

In the early stages of implementing Netcool Reporter you must plan how to structure your users, groups, channels, and datasources in a way that is logical for your particular environment. You must also define the roles you want each of your users to play, in other words, which data resources, functions and tools you want them to be able to access within the reporting application. These decisions become more complex - and more critical - as you move reporting out from a single department to multiple audiences in your organization, or maybe even out to external companies and organizations.

Ease of Distribution

In structuring your users, groups, channels, and datasources, a primary consideration is ease of report distribution. When you have several individual users who all need to run or view the same reports, it may be logical to place them in a single group for ease of report publishing and distribution. An even simpler alternative is to set up a reporting channel, or category, that all of these individual users subscribe to. As new individuals join a logical grouping of report users, all they have to do is subscribe to the channel to begin receiving their reports. Once groups and channels are defined, then report designers can begin to publish reports out to these logical groupings of users. Reports can be distributed on either on ad hoc or a scheduled basis, depending on your report designers' preferences.

System Performance

A second consideration in setting up users, groups, channels, and datasources is performance of the reporting application. One of your underlying goals in setting up the user environment is to minimize the number of unnecessary or redundant queries being run against the relational database. One of the main ways this can be controlled is by choosing to distribute static report outputs (snapshots) rather than dynamic report queries (definitions). For example, if 17 people across different departments want to see a listing of critical events on a weekly basis, it is advisable to distribute a static, weekly report to this group of people, rather than a query that will need to be run against the database 17 separate times.

Another way to manage system performance is by ensuring that only the users who really need to see a particular report have permission to run the underlying query. If all report definitions are made available to all users, than users may run queries against the database that are irrelevant in terms of their informational needs. This, in turn, places an unnecessary processing burden on both the database and the reporting server.

Security

A third area to consider in setting up users, groups, channels, and datasources is security of the data within the reporting application. Netcool Reporter offers several tools that help to maintaining a secure, protected environment for the overall database, the underlying data objects, and the report outputs. Database security is established mainly through the use of datasources. A datasource is a single username and password by which multiple users of Netcool Reporter access the relational database. Typically, this datasource is granted restricted access to the database (for example, select privileges but no alter privileges), and can only access the relevant tables containing event data. Datasource access is typically established either by the on-site DBA or the Netcool Reporter administrator.

Access to individual data tables and fields within the Netcool Reporter database can also be restricted, providing a further layer of security. This access can be controlled in one or more ways through the database and the reporting application. A database-centric approach would be based on setting up different datasources with different access privileges within the Netcool Reporter data or other related tables. Based on their assigned datasource, reporting users would see different tables, views, or data columns within the relational database. This approach requires the participation of the on-site DBA.

Within Netcool Reporter the LOOKUP_TABLE and LOOKUP_COLUMN parameters can be set to restrict user access to specific tables and even specific columns within the database. For example, if you have added a Customer field to your ObjectServer that you do not want reporting users to be able to query and view in their reports, you can exclude this data through the LOOKUP_COLUMN parameter.

It also provides the ability to exclude data on a row-by-row basis, through the use of parameters and/or the current user setting, both accessed through the Conditions tab of the Report Designer. For example, if you had four regional NOCs and you only wanted each region to see data pertaining to its own NOC, you could create a parameter on NOC (which might equate to the Node, Location, or OwnerGID field, for example). One option is to run each NOC report centrally and to generate a specific, parameterized report for each region. Another option is to set restrictions through the current user tool, which allows each NOC to run their own report but to select and view only data from their own region.

Managing External Audiences

Performance and security considerations come to the foreground even more when you want to extend your reporting application to audiences outside your own company or organization. Your external customers can, in most cases, log into your reporting application just like any internal client user (unless a firewall or another network security measure prevents this). As long as they have a web browser, the URL for your reporting server, and an assigned username and password, they can launch the portal and view any reports that have been published or distributed to them.

However, for reasons of data confidentiality, database performance, or customer management, you probably do not want to grant external audiences unrestricted access to your Netcool event data, relational database, and reporting application. Instead, you must define where these external users reside, from completely unrestricted through to tightly managed access. At the unrestricted end of the spectrum, external users can log into the portal, access report definitions, run reports against the live data, perhaps even launch the report designer tool and build their own queries.

Tighter controls can be placed on external access by prohibiting direct access to the database, distributing report snapshots rather than report definitions, setting up tightly managed channels for report distribution, or prohibiting use of the Report Designer. External users would retain some flexibility to view and perhaps even build reports, but would not be given free reign to access the raw data, run ad hoc reports, or view other peoples' reports.

At the other extreme, a highly secure and controlled reporting environment would mean building and running all reports internally, and perhaps not allowing external users to launch the portal. Instead, report outputs would be distributed as static files using external distribution methods, for example, e-mail or print distribution.

Tip 4: Format Reports for Maximum Impact

When building a customized report on your Netcool event data, you may wish to dedicate time not only to ensuring the right report content (for example, which tables, which columns, and what measures), but also to selecting and enhancing the appropriate report format. The Report Designer, HTML Customizer, and Freeform Report Designer all provide tools for selecting, modifying, and previewing report formats before they are distributed to other users.

Selecting The Right Report or Chart Type

When starting a new report, spend a little time visualizing how you want the report or chart to look when it is distributed. You may also determine that you want the same data to be viewed in multiple ways by end users, which is easy to do either by grouping separate report outputs into a presentation, or by launching multiple reports through a single HTML form. For example, note how the predefined report Events by Node actually consists of both a summary chart and a detailed tabular report, both of which can be launched simultaneously by a single HTML form.

Netcool Reporter includes several chart format options, ranging from 3D stacked bar charts to 2D line charts to colorful pie charts. Tabular report formats include drill-down reports, break reports, and even summary charts that break down into tables. The best format depends on the objective of the report. 3D charts lend themselves well to topological analysis, for example, “Where are the biggest problems in my network?”. 2D bar charts provide a simple, visual way to quantify and compare data across several variables, for example, “Which nodes are generating the most events, and at what severity level?”. Tabular reports, especially drill-down reports, can help organize a user's search through large volumes of data and point them where they need to drill down for more detail.

Formatting the Selected Report or Chart Type

Once you have selected the best output type(s), you may now wish to format the overall look and feel. Netcool Reporter includes several tools for customizing fonts, colors, size, orientation, sort order, and so on, for both tabular and graphical reports. Another way to format and customize charts is by using the Visual Customize tool located within the top menu of chart applets.

After you have run a chart, you can use this tool to further modify the chart layout, orientation, and so on. Once you have these settings defined, you can save them as a chart descriptor, which can now be applied in the future against that chart plus other, similar charts. The chart descriptor you have saved show up in the Report Designer under the Properties function the next time you launch the tool.

Finally, you can apply style templates, chart descriptors, and HTML templates in order to improve the appearance of your reports and charts.

Exporting to Other Applications

The reports and charts you generate may be stored and displayed as HTML, CSV, Excel 97, PNG, PDF, or XML. Selecting the right application format depends on the report audience and how they use the report outputs.

Some audiences may wish to receive a CSV or Excel file so they can perform their own spreadsheet manipulation of the data.

Recipients may prefer a static PNG or PDF file that requires little effort on their part to analyze and interpret.

Row based XML and Column Based XML may be available for reports depending on your user settings. Row-based XML may also be available for Crosstab reports. XML can be stored as data or transformed via an XML compiler.

Calling URLs from Reports

You can call a URL after the creation of the report using the Custom Call feature. This is a part of the report generating process.

Custom Calls are available for tabular reports, charts, lcharts, crosstabs, and cubes.

The Custom Call allows for other external actions to be triggered upon report creation. The URL can be any standard URL that can be called within a browser. The URL can also include parameters to pass to other applications.