CSM GUI on Linux Strategy White Paper

Author: Bruce Potter, Document Version 1.6, 1/5/05

Introduction

Initially, the user interface for CSM on Linux was only a command line interface, with the exception of the DCEM GUI for dsh. There are many factors to consider in choosing the direction for GUIs for CSM on Linux. This white paper briefly discusses all those factors and gives the rationale for a strategy.

Requirements & Guiding Principles

Before developing a strategy, it is important to understand the requirements that should be satisfied. These are the perceived requirements for a GUI for CSM on Linux, based on customer interaction with CSM, WebSM, and Perspectives, and based other GUIs in the Linux environment.

• Fast - our experience has shown that interfaces that start up quickly and allow the user to get right to their task are accepted more readily on UNIX platforms. This has been the case with SMIT and Linuxconf and was also true to some extent of the spmon GUI. This feature is even more important on Linux, which has been built on the fundamentals of small, fast, and efficient. In this environment, simple and fast interfaces like linuxconf and webmin have been very successful. Most users do not use GUIs exclusively, but do many things via the command line. When a GUI exists that helps them do a specific task more easily than they could do it from the command line, they will typically bring up that GUI, perform their task, and exit the GUI.
• Little Abstraction - linux administrators tend to be highly skilled and understand how the underlying components work. I don't think they are usually looking for a Microsoft-like interface that covers everything up and only presents an abstraction of icons representing objects. In our case, I think that means letting CSM show through and making the graphical interface similar to the command line interface, except with additional instruction, presentation, and tips to lead them through the common tasks, and combining multiple commands into single clicks so that the tasks can be done more easily.  It should even help the administrator learn the command line interface by optionally showing what commands are executed by the GUI, if possible.  (Note:  it is not always possible to efficiently implement all aspects of a GUI using command line interfaces, but this should be the goal.)
• Task-oriented - because we expect the user to use a combination of command line interfaces and graphical interfaces, the GUI should be organized with common tasks at the top level, instead of making them navigate to the correct list of objects before performing the task. This facilitates getting in, doing the task, and getting out quickly. This usage pattern also implies that the GUI needs to make the tasks sufficiently easier than the command line interface to motivate the user to bring up the GUI instead of doing the task with the command line. For tasks that we can't make substantially easier in the GUI, we shouldn't bother providing that task in the GUI. The primary tasks in CSM that should be supported are:
  • Monitoring all the nodes or a subset of nodes for conditions and performance values of interest.
  • Modify or add conditions and responses, and monitor those conditions.
  • Control or query the hardware of nodes.
  • Run commands on nodes.
  • Modify attributes of nodes.
  • Modify configuration files and distribute them to the nodes.
  • Install a new or existing node.
  • Update software on the management server and nodes.
  • Diagnose problems on nodes or the management server.
• Scalable - the GUI not only needs to perform well enough to display large amounts of information, but it also needs to provide mechanisms that allow the user to specify the subset of information they want to see.
• Available Anywhere - often administrators are on call all the time. When a problem occurs they need to be able to access the system from where they are. The GUI should be accessible from a Linux, Windows, or Mac desktop, and it should also work acceptably over a phone line.
• Low Resource Investment - we can't afford to invest a large number of resources in developing this interface, because most of our development resource needs to go toward increasing the capabilities of base CSM.  We need to choose an approach that results in a usable interface (even if it isn't the slickest) for a low cost.
• Open - if this interface is successful (i.e. used by many CSM administrators), then it will be important for people other than CSM development to be able to contribute to it. GUIs are typically resource intensive to develop because there are so many variations on the interaction with the user, and customers often have a lot of opinions about features that should be added to GUIs. Because of all the other features that need to be developed in CSM (high availability, scalability, PSSP transition, WebSM plugins for CSM, SMIT panels, etc.) it will be impossible to allocate enough resource to the CSM GUI on Linux to add all the features customers want. If the GUI can be made open source such that customers and other IBM development organizations can contribute features to it, it will come a lot closer to providing all the features users want. This GUI would only be useful to customers that have purchased CSM, so we would not be giving away any value by open sourcing it, and competitors would not be able to use it in their own products. One prerequisite of open sourcing the GUI is that its implementation has to be relatively simple so that potential contributors can easily understand how it works.  Another prerequisite to open sourcing it is that it can't use any IBM proprietary infrastructure.
• Work with Other IBM GUIs - to the extent that is possible, this GUI should somehow integrate with the other relevant IBM system management graphical interfaces:
  • IBM Director
  • xSeries service processor web interface
  • DCEM
  • WebSM interface for CSM on AIX
  • Virtualization Engine Console
• Positioned for PDA Support - although this has not received much attention yet, PDA access to CSM will become more important in the next couple of years. More administrators will use PDAs to monitor their systems and do a subset of operations on them. Our strategy must provide for a way to support this.

Proposed Strategy

This section briefly describes a proposed direction for the GUI for CSM on Linux that tries to take into account all the requirements above and satisfy them as much as possible.

The strategy is to create a simple web browser interface to CSM using CGI, perl-mod, and perl scripts on the back end that produce HTML and Javascript to be displayed in a browser. A web browser interface is well suited for simple, straightforward interfaces (because it is hard to develop a slick, fancy browser interface). If kept simple, they load quickly, can be run from any machine and can perform acceptably over a phone line. The forms-like interface lends itself to task oriented interfaces, and the technology chosen is open and familiar to many Linux developers. There are also some smaller advantages to this approach. For example, any HTML editor can be used to lay out the interface, the help system can simply be done in HTML, and links can be placed throughout the interface and help pages to the many web resources we are creating for CSM (e.g. documents, FAQ, downloads, etc.). Also, this technology positions us to support PDAs in the future. If we correctly modularize the back-end code that is producing the HTML, a module can be written that produces HTML appropriate for display on a PDA.

A version of this interface is under development can be downloaded from the CSM Fixes site. Comments and suggestions for this interface are welcome and can be sent to the CSM mailing list.

We are currently building this web interface as a module within Webmin. Webmin is an open source, very lightweight, web-based, system management interface infrastructure and set of applications. It is well known in the Linux community and using it gives us the following advantages:

• It installs quickly and easily and provides its own mini http server so that we don't have to modify the apache server on the CSM management server machine, which the customer may be using for something else.
• It is a familiar framework for the Linux community and provides many single system management applications that compliment the clustering features of CSM.
• It provides many useful perl functions for implementing a perl CGI program.
• It provides user authentication (login) and ACL control.
• It provides mechanisms for defaults, preferences, and context-sensitive help.
• It provides facilities for logging of actions performed on the system.

Why Not Use the WebSM CSM Interface on Linux?

This is a common (and valid) question so I will try to address it here. Although WebSM is a very full-featured GUI and the framework already runs on Linux (for the HMC), it does not satisfy several of the requirements listed above:

• It runs a little sluggishly and has a long start up time, which hinders it from being used for isolated tasks.
• It uses a Microsoft look and abstraction model, which will isolate users from the command line equivalent of operations.
• The console portion can run on Linux and Windows, but must be installed there first. It can not run on a Mac or run well over a phone line.
• It can't be open sourced because the implementation is too big and too complicated and is based on a proprietary infrastructure.
• Currently, Director and WebSM applications are not integrated, and given the size of both of the infrastructures it would be a sizable effort to do so.

Despite these disadvantages, it still might make sense to provide the WebSM CSM interface on Linux under certain circumstances. For example, if a customer has both AIX and Linux CSM management servers, they may want to use the same GUI on both of them. Also, if the HMC becomes a CSM management server at some point, it would be appropriate to provide the same GUI for the HMC functions and the CSM functions. But even if WebSM is provided on Linux in these cases, it still makes sense to provide the proposed web interface on Linux as an alternative for customers to use, since it has a very different design point that may appeal to customers that might not tend to use WebSM.

Integration with Other IBM GUIs

Probably the hardest requirement to satisfy is to make the GUI work in some sensible way with the other related IBM system management GUIs. This is difficult because some of them don't work with each other and are based on very different technologies. Let's take them one at a time:

• IBM Director - Director is a system management tool for xSeries servers. It runs on both Windows and Linux. The CSM web interface could be launched from the Director console and is probably the best approach beside specifically creating a Director extension, because the Director console can be run on any Windows or Linux machine and can be separate from the Director management server or the CSM management server. Since the Director console will know the location of the CSM management server, it can start a browser locally and point it at the URL of the CSM management server. Director already does this to launch the service processor web interface for a particular machine. We can also support launching the CSM web interface for a subset of nodes that the task is dropped on. The node list can be passed to the CSM web interface via arguments in the URL.
• xSeries Service Processor Web Interface - The service processor web interface is run by pointing the browser at the hostname of the service processor (RSA) of a machine. It provides hardware control and query functions for that machine and for other machines whose service processors are connected to this one. Since this is a web interface, it can be easily linked to from the CSM web interface.
• DCEM - This is a Java application that runs commands on all or some of the CSM nodes via dsh and displays the results for each node in a very nice way. It must be run on the CSM management server. If the user has an X server running on their desktop, the CSM web interface could launch DCEM on the CSM management server and remotely display it to the user's desktop.
• WebSM interface for CSM on AIX - WebSM is the system management GUI infrastructure for AIX. Applications (plugins) can be created for WebSM and a series of plugins for CSM have been developed. This is the primary interface to use if the CSM management server is an AIX machine. The CSM web interface proposed in this document would be the primary interface to use if the CSM management server is a Linux machine. Both of these interfaces support many of the CSM operations to mixed (both AIX and Linux) nodes because most of these operations are done via the management server. It is probably not important for these interfaces to be able to launch each other because it will probably be a while before we have customers that have both AIX and Linux management servers. But this can be considered in the future.

Virtualization Engine (VE) Console

This is the strategic direction for system management GUIs for eServer.  VEC is a browser-based console framework based on JSP portlets.  The eServer strategy says that all future system management GUIs should be portlets that fit into this framework.  This gives a single console, with a consistent look, for many applications that our customers interact with.  So how does the strategy outlined in this paper mesh with the eServer strategy?  In the following ways:

• Developing a interface based on webmin gives us a proven framework right now to deliver a browser-based GUI in the near term and gain user interface experience of how to best design pages for a web interface for CSM (what function should be grouped together on which pages, etc.), which can be quite different from a traditional GUI.
• At the proper time, portlet wrappers can be written to display the CSM web interface in VEC, with minimal changes.  Over time, the pages can be converted to JSP (from CGI).  Most of the HTML, Javascript and CSM commands would still apply.
• We should not invest significant development resource in VEC portlets for CSM until the technology is completely proven.  By this, I mean it has been delivered in other products, it is fast, it doesn't require too much other software to be installed, and the development environment is not too complicated.  There is good reason to be skeptical about ISC until it has proven itself, because there has been several other eServer or corporate GUI strategies that were killed or changed significantly before ever being delivered.  While we are waiting, it is reasonable to invest in a very low cost web interface implementation, in which some of the technology will be transferable to VEC.

Why Bother Developing a Lightweight Browser Interface for CSM on Linux?

With integration with Director and VEC coming, and the fact that many Linux administrators are satisfied with only a CLI, why bother developing a browser interface for CSM?  There are several reasons:

• It provides a learning tool for CSM.  The web browser interface is organized in a similar way to the CSM command line interface, and it has an option to show the user all the commands it is invoking.  This helps the user learn the CSM commands in a context that guides the user with forms, buttons, menus, explanation text, hints, and links to documentation.
• It provides a way to demo and present CSM.  CLIs don't demo well, because the audience does not get a quick, visual overview of what the product can do.  This web interface provides that.
• It shows how to do more complex operations with CSM.  Tasks like setting up monitoring conditions and responses, listing/filtering the AuditLog, and monitoring the performance of the cluster are more difficult to explain and do with the command line interface.  The web interface guides the user through these tasks and automates many steps of it.
• There is a small, but growing percentage of Linux administrators that appreciate a GUI in addition to a CLI.  As Linux transitions from early adopters to mainstream, less expert administrators will be using CSM.  A web interface is useful to this set of people.
• It will be a while before CSM and Director are converged enough that all the CSM operations can be done from the Director console.  This web interface fills the gap in a low cost way until that time.

Top 10 Reasons to Use the CSM Web Interface...

13.  (OK, so we had more than 10...)  All your friends are using web interfaces for system administration and monitoring.

12.  The browser paradigm is familiar to you:  Forms, the Back button, bookmarks, online shopping, dating services....  You get the idea.

11.  You can run it from anywhere:  a Windows, Linux, or Mac desktop or notebook, from home over a phone line, DSL, or cable modem, the other side of the world, etc.

10.  It contains everything you need to know to do your administration tasks:  plain English explanations, hints & tips, links directly to relevant documentation, etc.

9.  It helps you learn the CSM commands.  In an organized interface, it shows you all the features that are available to you in CSM, and with the verbose preference turned on, it will show you all the CSM commands it is running.

8.  You can bookmark any page in the web interface, so you can later jump directly to that page.  And since you probably already have a web browser running, it starts up super fast.  This facilitates using the CSM command line interface for most common operations, and jumping to the web interface for complex or less frequently done operations.

7.  It has 2 full pages of diagnostic operations, hints, and links to assist you when things go wrong.

6.  All those confusing arguments for creating Conditions and Responses are explained and displayed in a simple form.

5.  You can monitor all the conditions in your cluster with a smooth, efficient, patent pending, web page update method.

4.  It automates the mysterious incantations necessary to filter out uninteresting entries from the AuditLog, so you don't accidentally list 20,000 entries.

3.  You can monitor the progress of multiple installations with progress bars and one line status fields, instead of bulky consoles.

2.  With the click of a button, you can monitor the performance of your nodes and see historical graphs of performance data.

1.  It is small (less than 200K !), fast, and open.  Since it is all perl code, you can see all the source, see how it does things, and contribute improvements to it.

Developing the Web Interface

Basics

Contributing to the development of this web interface is pretty straight-forward once you understand the basic approach that is being used:

• Each page starts with a CGI perl program that should have the .cgi extension.  This perl script calls several functions from our GuiUtils perl module and from the Webmin library.  These functions parse the CGI parameters, create the standard page header and footer, etc.  Some of this is boiler plate and can be copied from any of the existing CGI scripts in this web interface.
• The purpose of the CGI script is twofold:
  • Examine the CGI input parameters to decide what this page should do.  These come as arguments on the URL and are the form data from the previous page.
  • Run any necessary CSM commands and set up the data in variables that will be accessed by the HTML.
• Then the CGI script usually calls GuiUtils->processHTML().  This function dynamically pulls in an HTML file, does some processing on it (described in a moment), and prints it to stdout.  (In CGI, the new page is formed by writing HTML to stdout.)  The processing done to the HTML file is:
  • All the HTML statements are interpolated in the CGI perl script context as if it had double quotes around it.  This means any variables starting with $ or @ will be replaced by their perl value.  (It also means any backslashes will cause the next character to have special meaning, which can be a problem if you have Javascript in your HTML that happens to use a backslash.  In this case, simply use 2 backslashes.)
  • Any text between "<!--CSM" and "CSM-->" will be executed (eval'd) as perl code.  This can be any perl code, but should also include print statements to produce HTML based on some logic.  Since "<!-- -->" is a comment in HTML, most HTML editors will ignore this code.
  • The previous 2 bullets mean that you can lay out your pages with any HTML editor and include in them perl variables and perl code for parts of the pages that need to vary based on your data collected in the CGI script.
  • The <head> </head> section of the file is stripped out, because Webmin adds its own header section to each page on the fly.
  • The HTML file can also include Javascript for any dynamic behavior necessary.
  • Also, all the HTML files should link to the external style sheet called style.css, which contains some common styles used by the interface.
• Webmin provides some infrastructure for help, preferences, ACL checking, and logging of actions.  These facilities should be used instead of implementing them yourself.
• The GuiUtils perl module provides functions for:  running commands, displaying messages, displaying verbose info, providing the common set of links at the top, etc.
• You can write javascript code in your HTML files to do some processing in the browser on the user's desktop.  Just remember that the perl code in your HTML file is run on the server (when your cgi file calls processHTML() ) and the javascript code is run on the user's desktop machine.  It is easy to get this confused when you are coding.

Gotchas

A few things that could be confusing at first:

• If you modify the cgi file for a page and then browse that page and get a Page Not Found error from the browser, you probably have a syntax error in your perl cgi code.  Tail /var/webmin/miniserv.error to see the error.
• The javascript code in your HTML files is run through the perl eval statement.  This means that a few special characters (e.g. \ and $) will be interpreted by perl.  So if you want an actual \ or $ to be in your javascript code, you must escape it with \.
• If your javascript code has a syntax error in it, IE will just stop executing that whole section of javascript.  If you don't have "Display notification about every javascript error" checked in the Advanced tab of Options, you won't see any error msg.

References

Having the correct information about all these technologies is key.  Here are some references:

• HTML
• Style Sheets
• Webmin programming
• Javascript tutorial and Javascript topics
• The O'Reilly book "Javascript: The Definitive Guide" is available at Amazon and the O'Reilly Online Safari Bookshelf
• CGI Info from NCSA and CGI Programming 101
• The O'Reilly book "CGI Programming with Perl" is available at Amazon and the O'Reilly Online Safari Bookshelf

Conclusion

The proposed strategy and approach satisfies most of the requirements as we understand them.  Comments, opinions, and suggestions are welcome and can be sent to the CSM mailing list.