www.alphaworks.ibm.comwww.ibm.com/developerwww.ibm.com

Home

Readme
Download

Build Instructions







Migration

Releases

Feedback

Y2K Compliance


CVS Repository
Mail Archive

Questions
 


Building XML4C with ICU using bundled Perl scripts on Windows
 

As mentioned earlier, XML4C may be built in stand-alone mode using native encoding support and also using ICU where you get support over 180 different encodings. ICU stands for International Components for Unicode and is an open source distribution from IBM. You can get ICU libraries from IBM's developerWorks site or go to the ICU download page directly.

Important: Please remember that ICU and XML4C must be built with the same compiler, preferably with the same version. You cannot for example, build ICU with a threaded version of the xlC compiler and build XML4C with a non-threaded one.

There are two options to build XML4C with ICU. One is to use the MSDEV GUI environment, and the other is to invoke the compiler from the command line.

Using, the GUI environment, requires one to edit the project files. Here, we will describe only the second option. It involves using the perl script 'packageBinaries.pl'.

Prerequisites:

  • Perl 5.004 or higher
  • Cygwin tools or MKS Toolkit
  • zip.exe

Extract XML4C source files from the .zip archive using WinZip, say in the root directory (an arbitrary drive x:). It should create a directory like 'x:\xml4c-src-3_5_1'.

Extract the ICU files, using WinZip, in root directory of the disk where you have installed XML4C, sources. After extraction, there should be a new directory 'x:\icu' which contains all the ICU source files.

Start a command prompt to get a new shell window. Make sure you have perl, cygwin tools (uname, rm, cp, ...), and zip.exe somewhere in the path. Next setup the environment for MSVC using 'VCVARS32.BAT' or a similar file. Then at the prompt enter:

set XERCESCROOT=x:\xml4c-src-3_5_1
set ICUROOT=x:\icu
cd x:\xml4c-src-3_5_1\scripts
perl packageBinaries.pl -s x:\xml4c-src-3_5_1 -o x:\temp\xml4c-3_5_1-win32 -t icu

(Match the source directory to your system; the target directory can be anything you want.)

If everything is setup right and works right, then you should see a binary drop created in the target directory specified above. This script will build both ICU and XML4C, copy the files (relevant to the binary drop) to the target directory.

For a description of options available, you can enter:

perl packageBinaries.pl

Building XML4C COM Wrapper on Windows
 

To build the COM module for use with XML on Windows platforms, you must first set up your machine appropriately with necessary tools and software modules and then try to compile it. The end result is an additional library that you can use along with the standard XML4C for writing VB templates or for use with IE 5.0 using JavaScript.

Setting up your machine for COM
 

To build the COM project you will need to install the MS PlatformSDK. Some of the header files we use don't come with Visual C++ 6.0. You may download it from Microsoft's Website at http://www.microsoft.com/msdownload/platformsdk/setuplauncher.htm or directly FTP it from ftp://ftp.microsoft.com/developr/PlatformSDK/April2000/Msi/WinNT/x86/InstMsi.exe.

The installation is huge, but you don't need most of it. So you may do a custom install by just selecting "Build Environment" and choosing the required components. First select the top level Platform SDK. Then click the down arrow and make all of the components unavailable. Next open the "Build Environment" branch and select only the following items:

  • Win32 API
  • Component Services
  • Web Services - Internet Explorer

Important: When the installation is complete you need to update VC6's include path to include ..\platformsdk\include\atl30. You do this by choosing "Tools -> Options -> Directories". This path should be placed second after the normal PlatformSDK include. You change the order of the paths by clicking the up and down arrows.

The order in which the directories appear on your path is important. Your first include path should be ..\platformsdk\include. The second one should be ..\platformsdk\include\atl30.

Building COM module for XML4C
 

Once you have set up your machine, build XML4C COM module by choosing the project named 'xml4com' inside the workspace. Then select your build mode to be xml4com - Win32 Release MinDependency. Finally build the project. This will produce a DLL named xerces-com.dll which needs to be present in your path (on local machine) before you can use it.


Testing the COM module
 

There are some sample test programs in the test/COMTest directory which show examples of navigating and searching an XML tree using DOM. You need to browse the HTML files in this directory using IE 5.0. Make sure that your build has worked properly, specially the registration of the ActiveX controls that happens in the final step.

You may also want to check out the NIST DOM test suite at http://xw2k.sdct.itl.nist.gov/BRADY/DOM/. You will have to modify the documents in the NIST suite to load the Xerces COM object instead of the MSIE COM object.



Building User Documentation
 

The user documentation (this very page that you are reading on the browser right now), was generated using an XML application called StyleBook. This application makes use of Xerces-J and Xalan to create the HTML file from the XML source files. The XML source files for the documentation are part of the XML4C module. These files reside in the doc directory.

Pre-requisites for building the user documentation are:

  • JDK 1.2.2 (or later).
  • Xerces-J 1.0.1.bundled
  • Xalan-J 0.19.2.bundled
  • Stylebook 1.0-b2.bundled
  • The Apache Style files (dtd's and .xsl files).bundled

Invoke a command window and setup PATH to include the JDK 1.2.2 bin directory

Next, cd to the XML4C source drop root directory, and enter

  • Under Windows:
    createDocs
  • Under Unix's:
    sh createDocs.bat

This should generate the .html files in the 'doc/html' directory.


I wish to port XML4C to my favourite platform. Do you have any suggestions?
 

All platform dependent code in XML4C has been isolated to a couple of files, which should ease the porting effort. Here are the basic steps that should be followed to port XML4C.

  1. The directory src/util/Platforms contains the platform sensitive files while src/util/Compilers contains all development environment sensitive files. Each operating system has a file of its own and each development environment has another one of its own too.
    As an example, the Win32 platform as a Win32Defs.hpp file and the Visual C++ environment has a VCPPDefs.hpp file. These files set up certain define tokens, typedefs, constants, etc... that will drive the rest of the code to do the right thing for that platform and development environment. AIX/CSet have their own AIXDefs.hpp and CSetDefs.hpp files, and so on. You should create new versions of these files for your platform and environment and follow the comments in them to set up your own. Probably the comments in the Win32 and Visual C++ will be the best to follow, since that is where the main development is done.
  2. Next, edit the file XercesDefs.hpp, which is where all of the fundamental stuff comes into the system. You will see conditional sections in there where the above per-platform and per-environment headers are brought in. Add the new ones for your platform under the appropriate conditionals.
  3. Now edit AutoSense.hpp. Here we set canonical XML4C internal #define tokens which indicate the platform and compiler. These definitions are based on known platform and compiler defines.
    AutoSense.hpp is included in XercesDefs.hpp and the canonical platform and compiler settings thus defined will make the particular platform and compiler headers to be the included at compilation.
    It might be a little tricky to decipher this file so be careful. If you are using say another compiler on Win32, probably it will use similar tokens so that the platform will get picked up already using what is already there.
  4. Once this is done, you will then need to implement a version of the platform utilities for your platform. Each operating system has a file which implements some methods of the XMLPlatformUtils class, specific to that operating system. These are not terribly complex, so it should not be a lot of work. The Win32 verions is called Win32PlatformUtils.cpp, the AIX version is AIXPlatformUtils.cpp and so on. Create one for your platform, with the correct name, and empty out all of the implementation so that just the empty shells of the methods are there (with dummy returns where needed to make the compiler happy.) Once you've done that, you can start to get it to build without any real implementation.
  5. Once you have the system building, then start implementing your own platform utilties methods. Follow the comments in the Win32 version as to what they do, the comments will be improved in subsequent versions, but they should be fairly obvious now. Once you have these implementations done, you should be able to start debugging the system using the demo programs.

Other concerns are:

  • Does ICU compile on your platform? If not, then you'll need to create a transcoder implementation that uses your local transcoding services. The Iconv transcoder should work for you, though perhaps with some modifications.
  • What message loader will you use? To get started, you can use the "in memory" one, which is very simple and easy. Then, once you get going, you may want to adapt the message catalog message loader, or write one of your own that uses local services.

That is the work required in a nutshell!


What should I define XMLCh to be?
 

XMLCh should be defined to be a type suitable for holding a utf-16 encoded (16 bit) value, usually an unsigned short.

All XML data is handled within XML4C as strings of XMLCh characters. Regardless of the size of the type chosen, the data stored in variables of type XMLCh will always be utf-16 encoded values.

Unlike XMLCh, the encoding of wchar_t is platform dependent. Sometimes it is utf-16 (AIX, Windows), sometimes ucs-4 (Solaris, Linux), sometimes it is not based on Unicode at all (HP/UX, AS/400, system 390).

Some earlier releases of xerce-c defined XMLCh to be the same type as wchar_t on most platforms, with the goal of making it possible to pass XMLCh strings to library or system functions that were expecting wchar_t paramters. This approach has been abandonded because of

  • Portability problems with any code that assumes that the types of XMLCh and wchar_t are compatible
  • Excessive memory usage, especially in the DOM, on platforms with 32 bit wchar_t.
  • utf-16 encoded XMLCh is not always compatible with ucs-4 encoded wchar_t on Solaris and Linux. The problem occurs with Unicode characters with values greater than 64k; in ucs-4 the value is stored as a single 32 bit quatity. With utf-16, the value will be stored as a "surrogate pair" of two 16 bit values. Even with XMLCh equated to wchar_t, xerces will still create the utf-16 encoded surrogate pairs, which are illegal in ucs-4 encoded wchar_t strings.

Where can I look for more help?
 

If you have read this page, followed the instructions, and still cannot resolve your problem(s), there is more help. You can find out if others have solved this same problem before you, by checking the Apache XML mailing list archives at http://archive.covalent.net and the Bugzilla Apache bug database.


Footer