|
Problem |
ORB may use TCP or Ephemeral ports to process requests for
Meta Operations or Full Value Descriptor (FVD) when tunnelling iiop
traffic from a client to a server over a firewall, depending upon
SendingContextRuntime service IP host information and
enableClientCallbacks runtime property values.
When a connection is established that the client side of the connection
uses a port number. Unless a client program explicitly requests a
specific port number, the port number used is an ephemeral port
number.
Ephemeral ports are temporary ports assigned by a machine's IP stack, and
are assigned from a designated range of ports for this purpose. When
the connection terminates, the ephemeral port is available for reuse,
although most IP stacks will not reuse that port number until the entire
pool of ephemeral ports have been used. So, if the client program
reconnects, it will be assigned a different ephemeral port number for its
side of the new connection. |
|
Cause |
In this case, the extra port access was due to meta
operations that are needed when classes have grown (between 1.4 and 1.3
several classes have grown and hence meta operations were needed to
enforce use of newer classes).
ORB uses the information provided in SendingContextRuntime service context
(that comes from the requesting side) to make meta operations. That
context will contain both the IP and port details.
ORB also exposes a property "com.ibm.CORBA.enableClientCallbacks"
(this will take true or false as values).
The logic of setting up SendingContextRuntime is as follows...
IF (enableClientCallbacks is true)
Setup SendingContextRuntime with the connections source port and IP
address.
ELSE
Use ORB's listener port to setup the SendingContextRuntime (if listener
port is configured it will use that value, else it will be an ephemeral
port). |
|
Solution |
Set com.ibm.CORBA.enableClientCallbacks=true
property in the client side (where the request originates) and verify
where the meta operations are targeted to afterwards, which should be
ideally the same port configured on Firewall for IIOP HTTP Tunnel Servlet.
This should effectively help to Restrict Firewall Port usage regarding
HTTP Tunneling Servlet ORB communications and prevent an entire range of
Ephemeral ports from being used or open at the Firewall, which increases
security and facilitates administration. There may be a slight trade-off
for performance obtained in using a range of preset Ephemeral ports for a
given IPStack, only comparative testing before and after mentioned
property has been set, would reveal to what extent.
Set com.ibm.CORBA.enableClientCallbacks=true on client side as jvm
parameter (-Dcom.ibm.CORBA.enableClientCallbacks=true ) and this will
cause the callback to be made on the same connection rather than trying to
establish a new one that the firewall would block.
Note that you can also configure static ORB Listener Address for the
Firewall by using:
-Dcom.ibm.CORBA.ListenerPort or with the ORB_LISTENER_ADDRESS endpoint
under the Administrative Console for the application server.
Customer Details:
===============
Details that may help to increase understanding of how and why this
problem manifests itself in production and how to diagnose such.
Tunnelling IIOP traffic from a client to the server over a firewall.
- J2EE fat client application running WebSphere® Application Server
Client 5 on an XP platform.
- WebSphere Application Server (5.1) running on Solaris 8. It is hosting
the server side of the application.
- In between there is a PIX firewall. The J2EE client lives on the
"inside" interface of the firewall. The Application Server lives on the
"outside" interface.
- IIOP is over HTTP by implementing tunneling via the
IIOPTunnelingServlet deployed to the Application Server. We have
configured the Tunneling Servlet to a certain port and have forced the
J2EE fat client to communicate through the IIOPTunnelingServlet.
- Allow traffic over the relevant web ports (including the tunneling
port) on the PIX firewall from any inside port to the specific outside IP
address and port number on the Application Server. Other traffic in this
direction has been denied. Logging is turned on for this rule.
- In the reverse direction, permit traffic from the relevant Application
Server ports (including the tunneling port), to any local port on the J2EE
fat client machine. All other ports from outside have been denied. Logging
turned on for this rule.
- Use secure authentication and configure
sas.client.props/soap.client.props with the appropriate client
certificates.
Problem Description:
==================
The behavior when there is no firewall is as follows:
- Run the application and we are prompted with a username/password
dialog. Enter the credentials of a user and are authenticated.
Onceauthenticated, the application displays a splash screen and a progress
meter showing the percentage of the application loaded. After the
application loads, the main screen is displayed. When you exit the
application it exists immediately.
- The behavior when the firewall is configured to pass all traffic is
the same as in 1. above.
- When implementing the firewall rules as described previously the
application reaches the splash screen (after authentication), but rather
than loading fully it halts at 63% loaded. It then hangs for 2-3 minutes.
Eventually it completes loading and runs as per step 1. and 2. When you
choose to exit the application it hangs for another 2-3 minutes before
finally logging out. At the points where it halts, it logs to a log4j file
detailing ORB errors. When we investigate the PIX firewall logs it reports
that it denied the request from the Application Server to establish a TCP
connection from a high numbered (ephemeral port) on the Solaris server to
a local ephemeral port on the J2EE thick client machine.
- When we relax the rule from the outside interface on the PIX firewall
so that it permits traffic from any port on the Application Server to any
port on the J2EE thick client machine, it loads completely as normal (as
per steps 1. and 2.)
Preliminary Diagnosis:
===================
The critical point seems to be point 3. above, under Problem Description.
The firewall shows that outbound traffic is being tunneled over the
tunneling port from the J2EE thick client to the WebSphere® Application
Server running on Solaris. However the firewall logs show that the
Application Server is attempting to open a TCP connection back to the J2EE
client over an ephemeral port, outside of the tunnel. This is expressly
denied in the firewall rules. The Log4J log below seems to suggest that
this traffic is ORB related.
The session eventually resumes but only after a 2-3 minute delay. It is as
if the Application Server is trying a direct connection back to the client
machine for a fixed period of time and then gives up.
Mismatched versions for client and server JDK result in callback requests
from server to client in the form of a meta request, to ensure use newer
classes and that this was previously done in a separate session and
currently done in same session. If the Application Server attempts to
contact the client to download updated classes for the client to
synchronize the server and client classes, attempt to open ephemeral TCP
port may occur when logic might imply or suggest that it should use the
same session and port already established through the tunnel ("is this a
leaky port"?). TCP Ephemeral ports are regulated by the OS for performance
reasons, but if we were using same session and abiding by set tunneling
parameter of "ALWAYS", then UnWanted TCP Ephemeral ports should never be
opened and used, as might be expected to occur when using "WHENREQUIRED"
tunneling parameter. However, using HTTP ForceTunnel "ALWAYS" is NOT
enough to govern that responses are processed over same port session
requests were initiated from. It simply implies that HTTP tunneling will
be used immediately without trying a TCP connection first. We now know
that one additional CORBA parameter is required to FORCE this logic if
desired, mainly the clientCallbacks being set to a value of TRUE.
- For all the requests initiated by the client, the callbacks (meta
operations or FVD callbacks) will then happen through the same connection
from which the request was made initially. This is the behavior is abided
by and observed when the com.ibm.CORBA.enableClientCallbacks=true
property is set.
- As of now, the property is available in all the current releases whose
build date is greater than 20040203. This includes Java™ 5.0 as
well. So the property is available in all available supported Java
versions (the IBM® Java versions) for WebSphere® Application Server
versions 5.0 and 6.0.
|
|
|
|
Cross Reference information |
Segment |
Product |
Component |
Platform |
Version |
Edition |
Application Servers |
Runtimes for Java Technology |
Java SDK |
|
|
|
|
|
|
|