Monday, September 14, 2009

Works Great Right Out of the Box

In reviewing my last two posts to this blog, I realized that I've actually been describing several of the SOA Platforms out-of-the-box actions in an ad hoc manner. I thought that this might be a good time to take a step back and review the platform's full set of out of the box actions.

Which got me thinking about how useful it is when a tool works "right of of the box" and how disappointing it can be when it doesn't.

In his book "Diary of a Wimpy Kid, The Last Straw,"[1] author Jeff Kinney has the main character, angst-ridden middle school student Greg Heffley, finds an ad for "personal hovercraft" in a comic book. Greg imagines himself as the coolest kid in school if he could get his own hovercraft, so he sends all his saved-up money away for one. He is, of course, disappointed when he opens the box and finds not a functioning personal hovercraft, but the plan to build one.

Step one in the plan reads: "Acquire an industrial twin-turbine engine."

In contrast, the SOA Platform provides an extensive set of actions[2] that work right out of the box. The SOA Platform also supports the creation of custom actions, but, when you are designing how your application will make use of the Platform, the place to start is the out-of-the-box actions. These have all been tested so you can depend on the results they will achieve, and they've also been documented and illustrated in the quickstart sample programs. (On a personal note, my introduction to the SOA Platform included my spending several hours building an action, when an out of the box action for the same task already existed!)

The out-of-the-box actions implemented in the SOA Platform are divided into the following functional groups:
  • Transformers and Converters - converting message data from one form to another
  • Business Process Management - integrating with JBoss jBPM
  • Scripting - automating tasks in supported scripting languages
  • Services - integration with EJBs
  • Routing - moving message data to the correct services
  • Notifier - sending data to ESB unaware destinations
  • Webservices/SOAP - the name says it all - support for Webservices
(There's also a Miscellaneous group that includes only one action - org.jboss.soa.esb.actions.SystemPrintln. This action prints of a message.)
Let's take a quick look at all of these actions, and then a more in-depth look at a couple of them.

Transformers and Converters

These actions are needed to enable services that expect data in different forms to communicate with each other. What transformers and converters do is to translate message payloads into different forms. Let's stop for a minute and review the contents of messages as they are passed over the JBossESB in the SOA Platform. Messages implement the org.jboss.soa.esb.message.Message interface. From the point of view of a service, the data in a message, in other words, the message "payload" consists of the message body plus attachments and properties. The default way for an action to get or set a message payload is the MessagePayloadProxy utility class. This class is important as it provides a standard way to get the message payload.

The actions in the Transformers and Converters actions group (yes, the names are pretty self-explanatory) are:
  • ByteArrayToString (org.jboss.soa.esb.actions.converters.ByteArrayToString) - This action converts a byte[] message payload into a Java String
  • LongToDateConverter (org.jboss.soa.esb.actions.converters.LongToDateConverter) - And this action converts a Java long message payload into a java.util.Date
  • ObjectInvoke (org.jboss.soa.esb.actions.converters.ObjectInvoke) - Things start to get a little more interesting with the ObjectInvoke converter. This action takes a serialized object, which constitutes the entire Message payload, and passes it to a configured processer class. The processed results become the new message payload. The action lets you use your own POJOs as the message payloads.
  • ObjectToCSVString (org.jboss.soa.esb.actions.converters.ObjectToCSVString) - This action converts a message into a comma separated string based on a list of property names that you supply.
  • ObjectToXStream (org.jboss.soa.esb.actions.converters.ObjectToXStream) - This action is similar to the ObjectInvoke action, except that it converts the object to XML using XStream[3]. You've probably already guessed that the XStreamToObject action converts XML to an object using XStream.
  • SmooksAction (org.jboss.soa.esb.smooks.SmooksAction) - This action enables you to use a powerful set of Smooks[4] operations within the SOA Platform. Transformations are probably the first type of operation that you think of with Smooks, but with the SmooksAction you can also make use of Smooks operations such as splitting and routing message payloads[5]. We'll take a closer look at this action later in the post. (Note that the SmooksTransformer action is being deprecated in favor of the SmooksAction action.)
  • PersistAction (org.jboss.soa.esb.actions.MessagePersister) - This action doesn't really transform or convert a message, it writes it to the persistent message store. The most common use of the action is that it is how the SOA Platform itself writes messages to the dead letter queue (DLQ), but it can be used in any situation when you want to store a message.
Business Process Management

This action group contains only one out of the box action (org.jboss.soa.esb.services.jbpm.actions.BpmProcessor), but that one action supports the JBossESB - jBPM integration in the SOA Platform. This integration enables your services to make calls into a jBPM process through the jBPM Command API. Of the commands in the command API, the following (3) are available for use from ESB:
  • NewProcessInstanceCommand creates a new ProcessInstance using a process definition that has already been deployed to jBPM. The process instance is left in the start state so that tasks referenced by start node are executed.
  • StartProcessInstanceCommand is the same as NewProcessInstanceCommand, except that the process instance that is created is moved from the start position to the first node in the process graph.
  • As its name implies, CancelProcessInstanceCommand cancels a process instance.
But, this is only part of the JBossESB - jBPM integration. The integration also supports the orchestration of services from jBPM processes. For a fuller description of the integration, see: http://jboss-soa-p.blogspot.com/2009/06/hanging-together-on-soa-platform.html

Scripting

This group of actions makes it possible for you to create custom actions using scripting languages. As of release 4.3 of the SOA Platform, scripting actions in Groovy and Bean Scripting Framework (BSF) are supported.
  • GroovyActionProcessor (org.jboss.soa.esb.actions.scripting.GroovyActionProcessor) - which uses Groovyscripts
  • ScriptingAction (org.jboss.soa.esb.actions.scripting.ScriptingAction) - which uses the Bean Scripting Framework and supports BeanShell, Jython and JRuby scripts
Both these actions accept the message payload and configuration data such as a Log4J logger as variables for the script that is executed by the action. In addition, they both support having scripting contained within the message itself.

Services

There's only one action in this group, but it's an important one.
  • The EJBProcessor (org.jboss.soa.esb.actions.EJBProcessor) action accepts a message and uses it to locate and invoke an EJB. (The SOA Platform supports both EJB2 and EJB3 through this action.) In order to find the correct EJB and invoke the correct method, you specify information such as the jndi-name, initial-context-factory, provider-url, method name and and other configuration specific to the EJB as message properties
Routing

One of the major tasks that the JBoss ESB in the SOA Platform performs is the routing of messages to services. (Remember that in an ESB, everything is either a message or a service.) In an earlier post, I described how to use JBoss Rules to perform content based routing (CBR), where the content of an incoming message determines its routing. The full set of routing actions are:
  • Aggregator (org.jboss.soa.esb.actions.Aggregator) - This is an interesting one. The Aggregator, as its name indicates, aggregates information together. This action enables you to combine information from multiple messages into one message. The Aggregator action is an implementation of the Aggregator Enterprise Integration Pattern[6].
  • EchoRouter (org.jboss.soa.esb.actions.routing.EchoRouter) - In contrast, the EchoRuter just logs and returns the incoming message.
  • HttpRouter (org.jboss.soa.esb.actions.routing.HttpRouter) - The HttpRouter routes the incoming message to a URL that you specify in the action definition.
  • JMSRouter (org.jboss.soa.esb.actions.routing.JMSRouter) - This action routes the incoming message to JMS. For example, to a JMS queue where a service can then retrieve the message asynchronously. In order to find the correct JMS queue or topic, you specify values for properties such as jndi-name, initial-context-factory, and jndi-URL in the action definition.
  • ContentBasedRouter (org.jboss.soa.esb.actions.ContentBasedRouter) - I described content based routing with JBoss Rules in an earlier post to this blog http://jboss-soa-p.blogspot.com/2009/07/when-content-knows-way-content-based.html). Content based routing is a flexible approach to message routing in that the content in a message is actually the determining factor in determining the route a message takes.
  • StaticRouter (org.jboss.soa.esb.actions.StaticRouter) - As its name implies, this router establishes static routes that do not change based on the content of the messages or a set of routing rules.
  • StaticWiretap (org.jboss.soa.esb.actions.StaticWiretap) - Maybe it's the comic book fan in me, but this is my favorite name for an action. There's something film noir-ish about a "wiretap." You can almost imagine Humphrey Bogart sitting it the back room of a bar listening in on a wiretapped SOA action. (On black and white film, of course.) In practice, it's not all that exciting. This action implements the Enterprise Integration Pattern for a wiretap[7]. The goal of a wiretap is to inspect each message, without affecting the operation of the action chain. This can be a useful action to use to aid in debugging a service.
Notifiers

Just as a gateway listener enables you to move ESB-unaware messages (in other words, messages in a format other than (org.jboss.soa.esb.message.Message) onto the SOA Platform's ESB, notifiers (org.jboss.soa.esb.actions.Notifier) enable you to move ESB-aware messages from the ESB to ESB-unaware services. The notifiers convert the ESB-aware messages into data into formats that these services can understand.

One thing to keep in mind is that the action pipeline has two stages, first normal processing and then outcome processing. Notifiers do not perform any processing of messages during that first stage. They send notifications during the second stage. The notification occurs after the processing of the action pipeline. This means that you cannot use notifiers to alter the results of any action processing. The data sent by the notifier is the ESB message that is processed by the action pipeline. The following notification targets (the names are very self-explanatory) are supported by the SOA Platform:
  • NotifyConsole - Prints the contents of the ESB message to the server log
  • NotifyFiles - Prints the message contents to a file or files
  • NotifySQLTable - Inserts the message contents into an existing database table
  • NotifyFTP - Prints the message contents to a file and then sends the file to a server via FTP
  • NotifyQueues and NotifyTopics - Writes the message contents into a JMS message and adds the JMS message to a specified JMS queue or topic
  • NotifyEmail - Sends the message contents in a mail message
Webservices/SOAP

When you're considering web services and the SOA Platform, you actually have two separate goals to achieve; you want to be able to expose webservice endpoints as services and you also want to be able to invoke web services from actions defined in a service.

There are (3) out of the box actions that support web services:
  • SOAPProcessor (org.jboss.soa.esb.actions.soap.SOAPProcessor) - This action is only available for use on the SOA Platform embedded server. You use this action to expose webservice endpoints. To do this, you write a "service wrapper webservice" that conforms to the JSR 181[8] ("Web Services Metadata for the Java Platform") standard. JSR 181 enables you to create webservices from a POJO by adding annotations such as @WebService and @WebMethod to it. This service wrapper webservice wraps calls to your service and exposes that service through JBossESB listeners. Your service can also be accessed through other transports (FTP, JMS, etc.) supported by the JBossESB in the SOA Platform. These 3 types of JBossWS webservice endpoints can be exposed through JBoss ESB listeners using this action:

    • Webservice bundled into a JBossESB .esb archive - When the webservice .war is bundled into a deployed .esb archive
    • Webservice bundled into an .ear - When the .war is bundled into a deployed .ear
    • Standalone webservice - When the webservice .war is deployed

  • WISE SOAPClient (org.jboss.soa.esb.actions.soap.wise.SOAPClient) - This action uses the WISE[9] client service (org.jboss.wise.core.client.WSService) to create a JAX-WS (Java API for XML Web Services) client class and then call the target service. WISE stands for "Wise Invokes Services Easily" and enables the dynamic generation of a client to access web services, without your having to write the code for the client. (This is referred to as "zero-code web service invocation").
  • SOAPClient (org.jboss.soa.esb.actions.soap.SOAPClient) - This action, uses the soapUI Client Service (org.jboss.soa.esb.services.soapui.SoapUIClientService) to create a message and then route that message to the target service.
Examples

We've already walked through how to use some of the out of the box actions (Business Process Management, Routing, Notifiers) in earlier posts to this blog. Now, let's look at some examples of other types of out of the box actions. As always, the "quickstart" programs in the SOA Platform provide the example code.

Transformation

We'll start by looking at transformations and the SmooksAction out of the box action. But first, let's take a look at Smooks. It's common to refer to Smooks as a transformation engine, but that's not the full story. Smooks is really a more general purpose processing framework that is capable of dealing with with fragments of a message. Smooks accomplishes this with "visitor logic", where a "visitor" is Java code that performs a specific action on a specific fragment of a message. This enables Smooks to perform different actions on different fragments of messages. As is stated by the Smooks project[10]:

Smooks supports these types of message fragment processing:
  • Templating: Transform message fragments with XSLT or FreeMarker
  • Java Binding: Bind message fragment data into Java objects
  • Splitting: Split messages fragments and rout the split fragments over multiple transports and destinations
  • Enrichment: "Enrich" message fragments with data from databases
  • Persistence: Persist message fragment data to databases
  • Validation: Perform basic or complex validation on message fragment data
The SOA Platform SmooksAction out-of-the-box action provides you access to all these Smooks capabilities.

Let's look at a very simple example, the aptly named "transform_XML2XML_simple" quickstart. This quickstart performs a message transformation by applying an XSLT (EXtensible Stylesheet Language Transformations) to an XML message. The message is transformed into XML in a different form. The interesting parts of the quickstart's jboss-esb.xml file are:

25  <actions mep="OneWay">
26 <action name="print-before" class="org.jboss.soa.esb.actions.SystemPrintln">
27 <property name="message" value="[transform_XML2XML_simple] Message before transformation" />
28 </action>
29 <action name="simple-transform" class="org.jboss.soa.esb.smooks.SmooksAction">
30 <property name="smooksConfig" value="/smooks-res.xml" />
31 <property name="reportPath" value="/tmp/smooks_report.html" />
32 </action>
33 <action name="print-after" class="org.jboss.soa.esb.actions.SystemPrintln">
34 <property name="message" value="[transform_XML2XML_simple] Message after transformation" />
35 </action>
  • Line 25: This line is not specific to transformations, but it's worth mentioning. "mep" stands for "message exchange pattern." The pattern used by this quickstart is "one-way" in that the requester invokes a service (by sending it a message) and then does not wait for a response.
  • Lines 26-28, 33-35: These lines simply cause the message to be written to the server log before and after its transformation. (org.jboss.soa.esb.actions.SystemPrintln, is, incidentally, the only out-of-the-box action in the Miscellaneous action group.)
  • Line 29: Here's where we specify that we want to invoke a SmooksAction
  • Line 30: And here is the XSLT that will be executed. We'll examine this file in a moment.
  • Line 31: This line is actually not in the quickstart. I've added the reportPath property it so that we can review the resulting report. Note that generating this report does require some processing resources, so it should not be used in production environments. See below for screen-shots of this report.





Now, let's look at smooks-res.xml:
1  <?xml version='1.0' encoding='UTF-8'?>
2 <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">
3
4 <resource-config selector="OrderLine">
5 <resource type="xsl">
6 <![CDATA[<line-item>
7 <product><xsl:value-of select="./Product/@productId" /></product>
8 <price><xsl:value-of select="./Product/@price" /></price>
9 <quantity><xsl:value-of select="@quantity" /></quantity>
10 </line-item>]]>
11 </resource>
12 <param name="is-xslt-templatelet">true</param>
13 </resource-config>
14 </smooks-resource-list>
  • Line 2: The namespace referenced here is the Smooks XML Schema Definition
  • Line 4: The resource-config element corresponds to an org.milyn.cdr.SmooksResourceConfiguration object[11]
  • Lines 7-9: These XPath (XML Path Language) [12] statements locate the Product element's productId and price attributes and the OrderLine element's quantity attributes. XPath is used by XSLT to find or reference data in XML documents.
When you run this quickstart, you'll see the XML message as defined in SampleOrder.xml displayed before and after it undergoes the XSLT transformation.

Web Services

Another important feature of the SOA Platform is its ability to publish web services as services. Remember how we talked about how the JBossESB included in the SOA Platform made it possible to deploy web service endpoints as endpoints on the ESB? The services can then also be accessed through other transports (FTP, JMS, etc.) supported by the JBossESB in the SOA Platform.

We'll look first at the webservice_producer quickstart. This quickstart demonstrates how to expose a JSR181 web service endpoint on the JBossESB within the SOA Platform by using the org.jboss.soa.esb.actions.soap.SOAPProcessor action.

Let's begin by looking at these lines in jboss-esb.xml:
37  <actions>
38 <action name="print-before" class="org.jboss.soa.esb.actions.SystemPrintln">
39 <property name="message"
40 value="[Quickstart_webservice_producer] BEFORE invoking jbossws endpoint"/>
41 </action>
42 <action name="JBossWSAdapter" class="org.jboss.soa.esb.actions.soap.SOAPProcessor">
43 <property name="jbossws-endpoint" value="GoodbyeWorldWS"/>
44 </action>
45 <action name="print-after" class="org.jboss.soa.esb.actions.SystemPrintln">
46 <property name="message"
47 value="[Quickstart_webservice_producer] AFTER invoking jbossws endpoint"/>
48 </action>
49 <action name="testStore" class="org.jboss.soa.esb.actions.TestMessageStore"/>
50 </actions>
  • Lines 38 and 45: As we discussed earlier, the SystemPrintln action displays the message to the server.log
  • Lines 42,43: Here is where we invoke the SOAPProcessor action to generate the JSR181 web service from the specified class, GoodbyeWorldWS
17  @WebService(name = "GoodbyeWorldWS", targetNamespace="http://webservice_producer/goodbyeworld")
21 @WebMethod
22 public String sayGoodbye(@WebParam(name="message") String message) {
23
24 Message esbMessage = SOAPProcessor.getMessage();
25 if(esbMessage != null) {
26 System.out.println("**** SOAPRequest perhaps mediated by ESB:\n" + esbMessage.getBody().get());
28 }
29 System.out.println("Web Service Parameter - message=" + message);
30 return "... Ah Goodbye then!!!! - " + message;
31 }
(Remember how we talked about having to write a "service wrapper webservice" that conforms to the JSR 181 ("Web Services Metadata for the Java Platform") standard? You do this by adding @WebService and @WebMethod annotations.)
  • Line 17: The definition of the web service.
  • Lines 24-31: The definition of the method in the web service to be invoked when the action is triggered.
The following lines in jboss-esb.xml are tangential to the goal of demonstrating exposing the webservice endpoint. But it is interesting as is shows some of the paths and protocols that can be used to inserting a message into the JBossESB withing the SOA Platform:
6   <providers>
7 <jms-provider name="JBossMQ" connection-factory="ConnectionFactory">
8 <jms-bus busid="quickstartGwChannel">
9 <jms-message-filter dest-type="QUEUE" dest-name="queue/quickstart_webservice_producer_gw"/>
10 </jms-bus>
11 <jms-bus busid="quickstartEsbChannel">
12 <jms-message-filter dest-type="QUEUE" dest-name="queue/quickstart_webservice_producer_esb"/>
13 </jms-bus>
14 </jms-provider>
15
16 <jbr-provider name="JBR-Http" protocol="http" host="localhost">
17 <jbr-bus busid="Http-1" port="8765" />
18 </jbr-provider>
19
20 <jbr-provider name="JBR-Socket" protocol="socket" host="localhost">
21 <jbr-bus busid="Socket-1" port="8888" />
22 </jbr-provider>
23 </providers>
30 <listeners>
31 <jms-listener name="JMS-Gateway" busidref="quickstartGwChannel" is-gateway="true"/>
32 <jbr-listener name="Http-Gateway" busidref="Http-1" is-gateway="true"/>
33 <jbr-listener name="Socket-Gateway" busidref="Socket-1" is-gateway="true"/>
34 <jms-listener name="JMS-ESBListener" busidref="quickstartEsbChannel"/>
35 </listeners>
The last quickstart we'll look at is webservice_consumer1 - as you can guess by its name, this quickstart demonstrates how to access (or "consume") a JSR181 Web Service in an ESB action by using the org.jboss.soa.esb.actions.soap.SOAPClient action.

This quickstart requires that a web service be deployed for the SOAPClient to access. The webservice is very simple and is deployed in a .war. In HelloWorldWS.java we see the definition of the web service:
28  @WebService(name = "HelloWorld", targetNamespace = "http://webservice_consumer1/helloworld")
29 public class HelloWorldWS
30 {
31 @WebMethod
32 public String sayHello(@WebParam(name = "toWhom")
33 String toWhom)
34 {
35
36 String greeting = "Hello World Greeting for '" + toWhom + "' on " + new java.util.Date();
37
38 return greeting;
39
40 }
41 }

Pay attention to the "toWhome" parameter. We'll look for this in the web service request.

In MyRequestAction.java, we see that toWhome parameter added as the key in a hashmap for the msg body:
44  /*
45 * Convert the message into a webservice request map.
46 */
47 public Message process(Message message) throws Exception
48 {
49 logHeader();
50 String msgBody = (String) message.getBody().get();
51 HashMap requestMap = new HashMap();
52
53 // add parameters to the web service request map
54 requestMap.put("sayHello.toWhom", msgBody);
55
56 // The "paramsLocation" property was set in jboss-esb.xml to
57 // "helloworld-request-parameters"
58 message.getBody().add(requestMap);
59 System.out.println("Request map is: " + requestMap.toString());
60
61 logFooter();
62 return message;
63 }
  • Line 54 - The towhome parameter added as the key in a hashmap for the msg body. The key of the HashMap is an Object-Graph Navigation Language (OGNL)[13] expression that identifies the SOAP parameter to be populated with the key's corresponding value. OGNL is an expression language for getting and setting Java objects properties.
Then, MyResponseAction.java retrieves the message and displays the response - see lines 54-55:
44  /*
45 * Retrieve and output the webservice response.
46 */
47 public Message process(Message message) throws Exception
48 {
49
50 logHeader();
51
52 // The "responseLocation" property was set in jboss-esb.xml to
53 // "helloworld-response"
54 Map responseMsg = (Map) message.getBody().get(Body.DEFAULT_LOCATION);
55 System.out.println("Response Map is: " + responseMsg);
56
57 logFooter();
58 return message;
59 }

In jboss-esb.xml:

36  <actions mep="OneWay">
37 <action name="request-mapper"
38 class="org.jboss.soa.esb.samples.quickstart.webservice_consumer1.MyRequestAction">
39 </action>
40 <action name="soapui-client-action"
41 class="org.jboss.soa.esb.actions.soap.SOAPClient">
42 <property name="wsdl"
43 value="http://127.0.0.1:8080/Quickstart_webservice_consumer1/HelloWorldWS?wsdl" />
44 <property name="responseAsOgnlMap" value="true" />
45 <property name="SOAPAction" value="sayHello"/>
46 </action>
47 <action name="response-mapper"
48 class="org.jboss.soa.esb.samples.quickstart.webservice_consumer1.MyResponseAction">
49 </action>
50 <action name="testStore" class="org.jboss.soa.esb.actions.TestMessageStore"/>
51 </actions>
  • Line 41: The SOAPClient class makes the call to the webservice. This is the "zero-code web service invocation" that we mentioned earlier in the post.
  • Line 43: The reference to the web service's wsdl
  • Line 44: The parameter responseAsOgnlMap tells the JBossESB move the SOAP response data into that OGNL-based map and attach it to the message.
    Line 45: The reference to the method to be invoked in the web service.
Closing Thoughts

It can be daunting to attempt to create a new service based application from scratch. The SOA Platform's combination of out-of-the-box actions and quickstart can make this task much easier. Even without a jet turbine engine. ;-)

Resources

[1] Kinney, Jeff, Diary of a Wimpy Kid, The Last Straw, Amulet Books; 1st edition (January 2009), ISBN-10: 0810970686, ISBN-13: 978-0810970687

[2] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/4.3.GA/html/Programmers_Guide/chap-SOA_ESB_Programmers_Guide-Out-of-the-box_Actions.html

[3] http://xstream.codehaus.org/

[4] http://www.smooks.org/mediawiki/index.php?title=Main_Page

[5] http://www.milyn.org/javadoc/v1.0/smooks/org/milyn/container/plugin/PayloadProcessor.html

[6] http://www.enterpriseintegrationpatterns.com/Aggregator.html The patterns described at this site and in the corresponding book were created to assist designers and implementers in creating enterprise software solutions. The patterns are platform and language agnostic, so they can be very helpful across many systems.

[7] http://www.enterpriseintegrationpatterns.com/WireTap.html

[8] http://jcp.org/en/jsr/summary?id=181

[9] http://jboss.org/wise

[10] http://www.smooks.org/mediawiki/index.php?title=Why_Smooks_was_Created

[11] http://www.milyn.org/javadoc/v1.0/smooks/org/milyn/cdr/SmooksResourceConfiguration.html

[12] http://www.w3.org/TR/xpath

[13] http://www.opensymphony.com/ognl/

Acknowledgments

As always, I'd like to thank the members of the SOA Platform project for their help and timely review comments! Also, this post relies heavily on the extensive SOA Platform documents and the quickstarts.

Thursday, July 23, 2009

Another post picked up by DZone...

The recent post on Gateways and Notifiers in the SOA Platform was just reposted to DZone here:

http://soa.dzone.com/news/soa-platform-gateways-and

Sunday, July 19, 2009

SOA Platform Gateways and Notifiers

One of the strengths of the SOA Platform is how it enables you to connect legacy applications together. How are these connections made? To answer that question, we have to think about awareness.

In this context, "awareness" refers to whether or not your application clients and services are "ESB-aware" or, in other words, whether they can understand the message formats and transport protocols used by the JBossESB in the SOA Platform. If you (as a programmer) are ESB-aware, then you can simply code any new clients and services that you write to also be ESB-aware. ESB-aware clients and services communicate with each other using messages. ESB-aware services are identified using Endpoint References (EPRs). These ESB-aware messages take the form defined by the org.jboss.soa.esb.message interface.

An ESB-aware message [1] consists of the following components:

  • Header: the header information contains information such as the destination EPR, the sender EPR, and where the reply goes--general message-level functional information.
  • Context: additional information that further explains the message; for example, transaction or security data, the identity of the ultimate receiver, or HTTP-cookie-like information.
  • Body: the actual payload of the message.
  • Fault: any error information associated with the message.
  • Attachment: any attachments (additional files) associated with the message.
  • Properties: any message specific properties (for example the jbossesb.message.id property is a unique value for each message.

But...

What about cases where legacy clients and services that are not ESB-aware? This is suddenly getting complicated. It sounds like you will have to build lots of custom plumbing adapters to get all these combinations of clients and services to talk to each other. But don't panic and don't start building home-grown infrastructure. Supporting interoperability between different clients and services is one of the goals of the SOA Platform.

Interoperability through ESB adapters

One of the ways that the SOA Platform makes this interoperability possible is though gateway adapters. A gateway (org.jboss.soa.esb.listeners.gateway) is a service that acts as a bridge between an ESB-aware and an ESB-unaware client and service. Gateways translate information between ESB and non-ESB message formats and EPRs. (EPR stands for endpoint reference.) Gateways are listener processes in that they "listen" for incoming communications. They are different from ESB-aware listeners in that they accept data in different formats such as objects in files or SQL tables. ESB-aware listeners can only accept messages in the org.jboss.soa.esb.message format.

The SOA Platform supports these gateways:

  • file gateways: local filesystem, ftp, sftp and ftps
  • JMS
  • HTTP/HTTPS
  • email (POP3)
  • SQL table
  • Hibernate

When a gateway listener hears something (incoming data), it converts the non-ESB messages that it receives into the org.jboss.soa.esb.message format. How this conversion happens depends on the gateway type. For example, the file gateway takes the contents of the file and puts it into the location named "BytesBody.BYTES_LOCATION" within the message body.

Let's look at an example.

A quick start (with a quickstart)

One of the helpful features included in the SOA Platform is the set of "quickstart" sample applications. The quickstarts demonstrate SOA Platform functions and integrations. The goals of the quickstarts are to serve as learning tools and as a starting point for your own application coding. We'll examine one of these quickstarts to view the operation of a gateway.

Let's assume that you have a legacy application that stores data in text files. What you want to do is to create an ESB-aware service to read these files and process them as they are created. You could write infrastructure code that periodically polls a target directory to determine when a new file is deposited there. With the SOA Platform, however, you can simply use a file gateway.

The quickstarts are distributed under the samples/quickstarts directory. Let's take a look at helloworld_file_action.
cd samples/quickstarts/helloworld_file_action

Deploy the test to the server and run it with these simple commands:
ant deploy
ant runtest

The server log should look something like this:
16:42:22,744 INFO [STDOUT] Message structure:
16:42:22,744 INFO [STDOUT] [Hello World In A File].
16:42:22,746 INFO [STDOUT]
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
16:42:22,746 INFO [STDOUT] Body: Hello World In A File
16:42:22,746 INFO [STDOUT] &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
16:42:22,747 INFO [STDOUT] Message structure:
16:42:22,747 INFO [STDOUT] [ message: [ JBOSS_XML ]

What just happened here? Let's take a look. The "runtest" ant target invokes CreateTestFile.java program. As its name implies, this program creates a test file. The file contains this text: "Hello World In A File"

For what happens next, we need to look at how the test application is configured. A SOA Platform application's primary configuration file is named jboss-esb.xml. In the case of this quickstart, that file is generated at run time so that the SOA Platform install directory can be determined. So, we need to look at the source of the generated file, which is this file: jboss-esb-unfiltered.xml

The first part of the file that we're interested in looks like this:
1  <fs-provider name="FSprovider1">
2 <fs-bus busid="helloFileChannel" >
3 <fs-message-filter
4 directory="@INPUTDIR@"
5 input-suffix=".dat"
6 work-suffix=".esbWorking"
7 post-delete="false"
8 post-directory="@OUTPUTDIR@"
9 post-suffix=".sentToEsb"
10 error-delete="false"
11 error-directory="@ERRORDIR@"
12 error-suffix=".IN_ERROR"
13 />
14 </fs-bus>
15 </fs-provider>

The lines that we're most interested in are:

  • Line 4,5: This is the definition for the input file directory and extension that the listener "listens for."
  • Line 6: When the message in the file is being processed, a work file with this extension is created by the listener.
  • Lines 7-9: And, when that processing is complete, the message is written to a file in the output directory. The file is retained after the listener is complete.
  • Lines 10-12: Unless something goes wrong, then the message is written to an error file.

This is the definition of the file gateway that we'll use. Note the directory names. In this example, they are filled in at run time from values determined by the directory in which you installed the SOA Platform.

Now, remember how we said that everything is either a message or a service? Scroll down in the file and we'll see our service.
1  <service
2 category="myCategory"
3 name="myFileListener"
4 description="Hello World File Action (esb listener)" >
5
6 <listeners>
7 <fs-listener name="FileGateway"
8 busidref="helloFileChannel"
9 maxThreads="1"
10 is-gateway="true"
11 poll-frequency-seconds="10"
12 />
13 <jms-listener name="helloWorldFileAction"
14 busidref="quickstartEsbChannel"
15 maxThreads="1"
16 />
17 </listeners>
18
19 <actions mep="OneWay">
20 <action name="testAction" class="org.jboss.soa.esb.actions.SystemPrintln" />
21 <action name="action1"
22 class="org.jboss.soa.esb.samples.quickstart.helloworldfileaction.MyAction"
23 process="displayMessage,playWithMessage"
24 />
25 <action name="dump" class="org.jboss.soa.esb.actions.SystemPrintln">
26 <property name="printfull" value="true"/>
27 </actions>
28 </service>

Let's look at this service description.

  • Lines 7-12: First, the listeners. Note the definition of the file system (fs) listener. This is our gateway. The "busidref" field references the "fs-provider" that we discussed a minute ago. Did you notice a second listener? This is an ESB-aware JMS listener. It is there because the SOA Platform requires that each gateway have a corresponding ESB listener defined.
  • Lines 19: Let's look at the actions. The "mep" definition of "OneWay" refers to the action pipeline (the sequence of actions) explicitly not sending a response. (You can find more detail about these definitions in the SOA Platform Programmers' Guide [2].)
  • Lines 20, 25-26: The actions named "testAction" and "dump" write the message to the server log.
  • Lines 21-23: The action that we're most interested in is "action1." The methods (displayMessage, playWithMessage) in the class referred to by this action (org.jboss.soa.esb.samples.quickstart.helloworldfileaction.MyAction) receive the data in the input file as an ESB-Aware message, extract that information, and process it. In the case of your application, you would replace “MyAction” with your own business logic code.

So, in summary, what happens when this quickstart is run?

  1. The file system gateway is initialized.
  2. A file is created in the input directory defined in its fs-provider definition.
  3. The gateway reads the file, transforms it into an ESB-aware message, and passes the message to the methods in the action class defined in the action definition.
  4. These methods process the message (in the case of the quickstart, they write the message contents to the server log).5. The gateway moves the file to the output directory defined in its fs-provider definition.


It's important to note that the only new code that had to be written is the action class. The infrastructure code to listen for and transform the file into a message in the format that the SOA Platform can process is all part of the gateway.

From the perspective of an application developer, this is a plus as you can concentrate on writing code to fulfill the business process requirements of your application.

From the perspective of a QE engineer, this is a real plus as it means that less code has to be written. And, the less code, the fewer opportunities for bugs!

The Data is in the SOA Platform, Now What?

The gateway listeners make it possible for your legacy applications to get data into the SOA Platform's ESB and have that data transformed into ESB-aware messages, so that the ESB can route the data to the desired destination service. However, that's only have the battle if the destination service is another legacy application that is also not ESB-aware. How can you easily transform the messages into a form that a legacy application can process?

The answer is to use one of the SOA Platform's notifiers. Just as a gateway listener enables you to move ESB-unaware messages onto the SOA Platform's ESB, notifiers enable you to move ESB-aware messages from the ESB to ESB-unaware services. The notifiers convert the ESB-aware messages into data into various forms that your services can understand. The SOA Platform supports these notifiers:

  • NotifyConsole
  • NotifyFiles
  • NotifySQLTable
  • NotifyFTP
  • NotifyQueues
  • NotifyTopics
  • NotifyEmail

Let's add some of these notifiers to the helloworld_file_action quickstart. Note that while there are other quickstarts dedicated to illustrating notifiers, our expanding on the helloworld_file_action quickstart is in keeping with one of the goals of the quickstarts; to give you a starting point for developing your own customer applications.

One thing to keep in mind is that the action pipeline has two stages, first normal processing and then outcome processing. Notifiers do not perform any processing of messages during that first stage. They send notifications during the second stage. The notification occurs after the processing of the action pipeline. This means that you cannot use notifiers to alter the results of any action processing. The data sent by the notifier is the ESB message that is processed by the action pipeline. [3]

Let's insert the following statements (without the line numbers) into the service defined in the quickstart's jboss-esb-unfiltered.xml file:
1  <action name="notificationAction" class="org.jboss.soa.esb.actions.Notifier">
2 <property name="okMethod" value="notifyOK" />
3 <property name="notification-details">
4 <NotificationList type="ok">
5 <target class="NotifyConsole"/>
6 <target class="NotifyFiles">
7 <file append="false" URI="@OUTPUTDIR@/notify.txt"/>
8 <file append="true" URI="@OUTPUTDIR@/results.log"/>
9 </target>
10 <target class="NotifyEmail"
11 from="soauser@server.com"
12 sendTo="user@server.com.com"
13 ccTo="admin@server.com"
14 subject="Test was successful"
15 message="See attached file">
16 <attachment>@OUTPUTDIR@/notify.txt</attachment>
17 </target>
18 </NotificationList>
19 <NotificationList type="err">
20 <target class="NotifyConsole"/>
21 <target class="NotifyFiles">
22 <file append="true" URI="@ERRORDIR@/error.log"/>
23 </target>
24 </NotificationList>
25 </property>
26 </action>

Before we re-deploy and rerun the quickstart, we'll walk through these new statements:

  • Line 1: Note the reference to the Notifier class
  • Line 2: The okMethod enables the server to notify about success or failure at each action in the action pipeline
  • Line 4: These notifications will happen if the processing of the action pipeline is successful
  • Line 5: The simplest notifier - the ESB message is sent to the server console log
  • Line 6: Since we started the quickstart by creating a file, we'll generate another file with the NotifyFiles notifier.
  • Line 7: The ESB message is written to a file named "notify.txt" in the output dir. Note that if a file with this name already exists, it is overwritten.
  • Line 8: We'll also append the ESB message to a log file.
  • Line 10: And, we'll send that same ESB message in an email - the ESB message will be in the email
  • Line 11: But, just to be sure, we'll also send the notify.txt file as an attachment to the email
  • Line 19: Finally, if the action pipeline processing encounters an error, these notifiers will execute

At this point, redeploy the quickstart and execute "ant runtest" again.

The server log should look something like this:
 2009-07-18 22:45:33,291 INFO  [STDOUT] ConsoleNotifier 2009/07/18 10:45:33.291<
BEFORE
Hello World In A File
AFTER
>
2009-07-18 22:45:33,292 INFO [org.jboss.soa.esb.helpers.Email] Initialising mail server session. Properties: {mail.smtp.port=25, mail.smtp.auth=true, mail.smtp.host=localhost}

For the console and email notifier. An email will be sent to the addresses you specified and you should also see a file named notify.txt and results.log in the output directory. If you run the test again, you'll see the results.log file include the ESB message twice.

Closing Thoughts

With the SOA Platform's gateways and notifiers, it's possible to reuse existing applications as services that are ESB-unaware, without having to rewrite the applications' code. The applications can continue to communicate via files or FTP or database records or other data formats. The gateways and notifiers take care of getting data into the SOA Platform (where you can route it to the intended service - see the previous post on content based routing) and then back out. And, that is a lot easier than having to rewrite legacy applications. COBOL anybody? ;-)

Resources

[1] Look in this file in the javadocs installed with your SOA Platform server: jboss/soa/esb/message/Message.html
[2] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/4.3.GA/html/Programmers_Guide
[3] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/4.3.GA/html/Programmers_Guide/ch11s06.html

Acknowledgments

As always, I'd like to thank the members of the SOA Platform project for their help and timely review comments! Also, this post relies heavily on the extensive SOA Platform documents and the quickstarts. The information on listeners in this post was originally published in Red Hat Magazine (http://magazine.redhat.com/2008/05/22/adapters-for-an-esb). I'd like to thank everyone at the magazine for their help over the past few years, especially the magazine editor, Bascha Harris.

Tuesday, July 14, 2009

DZone Reposts Content Based Routing Post

The recent post on Content Based Routing with JBoss Rules in the SOA Platform was just reposted to DZone here: http://soa.dzone.com/news/when-content-knows-way-content?mz=3006-jboss

It's a bit recursive - posting a post about a post - but it's good to see this subject in DZOne too!

Monday, July 6, 2009

When the Content Knows the Way - Content Based Routing in the SOA Platform

In my last post to this blog, I examined the ESB-jBPM integration in the SOA Platform. This time, we'll take a look at one aspect of the integration with JBoss Rules.

Introduction

The routing of data from one place to another is one of the most basic, and common, problems facing any networked software application. This routing can take many forms, such as email being sent to the correct recipient or network traffic being routed around the globe based on system names defined in DNS.

In the context of an Enterprise Service Bus such as the JBoss ESB in the SOA Platform, where everything is either a message or a service, routing means getting messages delivered to the correct services. There are multiple ways to route data to a service. It's possible to define these routes statically, which can make sense for an application where some type of data is always directed to a set endpoint. But, this approach will fail if a destination service is unavailable or is moved. You can control the route that the messages take across the ESB in a number of ways. In this post, we'll examine routing messages based on message content with the content based routing pattern as illustrated in one of the SOA Platform "quickstart" sample programs.

JBoss Rules

One of the great challenges in developing business application software is the separation between the business logic, or the "rules" that you want to govern the application, and the technical programming tasks necessary to actually build the application. What's more, it can be expensive and difficult to maintain application code, and keep it in synch with constantly changing business conditions and while not destroying the original design and turning the code into a set of ever more complex if-then-else statements. What's needed is a mechanism to define the business rules and then execute the rules without having to hardcode the rules into the application code.

What's needed is a rules engine. JRS-94[1] defines the standard for a Java rules engine API. The standard defines the API to register, retrieve and execute rules. JBoss Drools[2] (referred to as JBoss Rules in the SOA Platform) is based on this standard, but more than just a rules API and rules programming language, Drools is a complete enterprise platform for rules-based application development, workflow, administration, and event processing. It also provides an integration with JBossESB to support content based routing.

Let's start by examining at the term "content based routing."[3] The routing part of the term is easy; we're talking about getting messages routed to the correct service. When we talk about "content based" routing, what we want to have happen is to have the ESB examine a message, and based on its content, select the correct routing path. But, we don't want to have the code to make these routing decisions built into the services or the ESB itself. We want to use a rules-based approach, where we can take advantage of the power and flexibility of a rules definition language to construct the decision making routing. We also want to take advantage of the efficiency of a rules engine to perform this routing, instead of coding complex and hard to maintain if-then-else statements into the application.

OK. It's time to look at a working example.

One of the great features of the SOA Platform is its extensive set of "quickstart" programs. These programs illustrate various features supported by the ESB. For our example, we'll look at the fun_cbr quickstart.

Like many of the quickstarts, fun_cbr starts by placing a message into a queue. A service listening to that queue then takes that message and sends it to a destination service. What we're interested in looking at in this quickstart, is how the content of that message determines the route that the message takes to one of three defined destination services.

Let's start by examining with the message and its content. When you run the quickstart, the "SampleOrder.xml" (for a mythical DVD store) is file is read into the message that is sent. The file looks like this:

In SampleOrder.xml:

1 <Order xmlns="http://org.jboss.soa.esb/Order" orderId="1" statusCode="0"
2 netAmount="59.97" totalAmount="64.92" tax="4.95">
3 <Customer userName="user1" firstName="Harry" lastName="Fletcher" state="SD"/>
4 <OrderLines>
5 <OrderLine position="1" quantity="1">
6 <Product productId="364" title="The 40-Year-Old Virgin " price="29.98"/>
7 </OrderLine>
8 <OrderLine position="2" quantity="1">
9 <Product productId="299" title="Pulp Fiction" price="29.99"/>
10 </OrderLine>
11 </OrderLines>
12 </Order>


Nothing in this content is that unusual (except perhaps for Harry's taste in movies). Make a mental note of the "statusCode" element on line #1. We'll come back to this in a bit.

OK, we have build a message that contains this content and place that message in a queue so that a service can receive it and execute an action on it. Now what?

Let's look at that action in the "jboss-esb.xml" file. (This file defines the configuration of, and the actions performed, by the quickstart.)

In jboss-esb.xml:

44 <action class="org.jboss.soa.esb.actions.ContentBasedRouter" name="ContentBasedRouter">
45 <property name="ruleSet" value="FunCBRRules-XPath.drl"/>
46 <property name="ruleLanguage" value="XPathLanguage.dsl"/>
47 <property name="ruleReload" value="true"/>
48 <property name="destinations">
49 <route-to destination-name="blue" service-category="BlueTeam" service-name="GoBlue" />
50 <route-to destination-name="red" service-category="RedTeam" service-name="GoRed" />
51 <route-to destination-name="green" service-category="GreenTeam" service-name="GoGreen" />
52 </property>
53 </action>


Let's examine this section of the file line-by-line:

Line 44: The org.jboss.soa.esb.actions.ContentBasedRouter class is one of the SOA Platform's predefined "Out-of-the-box Actions." The SOA Platform provides a set of these actions, that you can always augment by writing your own custom actions[4]. Before you write your own, you should take a look at the out-of-the-box actions as you may find one that meets your application's needs.

Line 45: Here's where we define the set of rules that govern the content based routing. Remember that in this context, the rules are defined as JBoss Rules. We'll examine these rules in just a minute.

Line 46: In order to be able to parse information out of XML data in a message, the SOA Platform includes a domain specific language (DSL) implementation to use XPath to traverse the XML. This is defined in the jboss-as/server/production/deploy/jbrules.esb/XPathLanguage.dsl file. If you're unfamiliar with XPath[5], it's really worth learning as it has many useful applications. For example, some GUI automation tools such as Selenium support using XPath to locate UI elements if you are unable to rely on the UI elements having static ID's. Also note that XPathLanguage.dsl supports both namespace specific and non-namespace specific syntaxes. In this quickstart, a namespace specific syntax is used.

Line 47: This property allows you to specify if the rules should be reloaded each time they are used. This has no effect on the small set of rules used in the quickstart, but it can cause a performance hit on a large set of rules. So, setting this to "true" enables you to modify the rules as defined in the copy of FunCBRRules-XPath.drl deployed to the server without having to redeploy the quickstart to the SOA-P server. Modifying the local copy of the rules file will not cause the rules to be reloaded. You have to update the drl file that is deployed with the quickstart.

Lines 49-51: These are the routes to the destination services.

Now it's time to take a look at the rules that are defined in FunCBRRules-XPath.drl

In FunCBRRules-XPath.drl:

1 package com.jboss.soa.esb.routing.cbr
2
3 #list any import classes here.
4 import org.jboss.soa.esb.message.Message;
5 import org.jboss.soa.esb.message.format.MessageType;
6
7 expander XPathLanguage.dsl
8
9 #declare any global variables here
10 global java.util.List destinations;
11
12 rule "Blue Routing Rule using XPATH"
13 when
14 xpathEquals expr "/order:Order/@statusCode", "0" use namespaces "order=http://org.jboss.soa.esb/Order"
15 then
16 Log : "Blue Team";
17 Destination : "blue";
18 end
19
20 rule "Red Routing Rule using XPATH"
21 when
22 xpathEquals expr "/order:Order/@statusCode", "1" use namespaces "order=http://org.jboss.soa.esb/Order"
23 then
24 Log : "Red Team";
25 Destination : "red";
26 end
27
28 rule "Green Routing Rule using XPATH"
29 when
30 xpathEquals expr "/order:Order/@statusCode", "2" use namespaces "order=http://org.jboss.soa.esb/Order"
31 then
32 Log : "Green Team";
33 Destination : "green";
34 end


Line 7: Here is the reference to the XPath definitions.

Line 10: The destinations global variable is the point of integration to the destinations defined in the jboss-esb.xml file.

The rules are all the same, except for the status code value, so we'll only examine one of them. (In the process, we'll walk through a short lesson in writing a rule.)

Line 12: The start of a rule definition.

Line 13: The start of the "when" construct of a rule. Each rule definition includes a "when" construct (the criteria that must be met) and a "then" construct (the action to take if the "when" construct is met).

Line 14: The XPath syntax translates to "starting at the root of the document, find an Order element with a statusCode attribute equal to 0."

Line 15: The then construct starts here.

Line 16: Generate a log message

Line 17: Add a destination's name to the global list called "destinations, " which is then evaluated by org.jboss.soa.esb.actions.ContentBasedRouter that invoked the rule.

If you're getting a little lost now, this diagram may shows how things are connected.
So what happens when the quickstart is deployed and run?

  • An incoming message is placed into the queue that is watched by the listener configured with the ContentBasedRouter action
  • That action is configured with the rule set defined in FunCBRRules-XPath.drl
  • The action class puts the message into the Rules' working memory and fires the rules
  • Based on the results of the rules, a list of destinations is created
  • And the message is sent to the services at those destinations - in the case of this test, the message is sent to the blue team


(There's actually a bit more to it for the incoming message. JBoss ESB actually routes ESB formatted messages to services. The ESB supports adapters to enable other formats for incoming messages. These adapters operate with "gateway" services to enable you to connect existing services to the SOA Platform.[6])

Closing Thoughts

As we discussed in the introduction, one of the great strengths of the SOA Platform is the set of integrations that it supports. With its integration with JBoss Rules, you can deploy Rules-based services to the SOA Platform server and utilize JBoss Rules for content based routing. With content based routing, the information in the messages themselves determine the messages' destinations.

References

[1] http://jcp.org/en/jsr/detail?id=94
[2] http://www.jboss.org/drools
[3] http://www.jboss.org/jbossesb/docs/4.3.GA/manuals/html/services/ContentBasedRouting.html
[4] http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/4.3.GA/html/Programmers_Guide/ch11s05.html
[5] http://www.w3schools.com/XPath/default.asp
[6] http://magazine.redhat.com/2008/05/22/adapters-for-an-esb

Acknowledgments

As always, I'd like to thank the members of the JBossESB (see http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/Contributors.txt), JBoss Rules projects, SOA Platform project - especially Burr Sutter, Mark Little, Mark Proctor and Jarek Kijanowski - for their help and timely review comments! Also, this article relies heavily on the extensive JBossESB and JBoss Rules user documents and the quickstarts.

Monday, June 15, 2009

Hanging Together on the SOA Platform - Introduction to the ESB-jBPM Integration

One of the great strengths of the SOA Platform is its wealth of software integrations. In this post, I'll examine the JBoss ESB - jBPM integration in the SOA Platform and how it can be used to connect business processes to services.

JBosssESB

Service Oriented Architecture (SOA) is not a single program or technology. It’s really a matter of software architecture or design. In hardware terms, a “bus” is a physical connector that ties together multiple systems or subsystems. Instead of having a large number of point-to-point connectors between pairs of systems, you connect each system to the bus once. An Enterprise Service Bus (ESB) does the same thing, logically, in software.

Instead of passing electric current or data over the bus to and from the connections (or “endpoints”) on the ESB, the ESB logically sits in the architectural layer above a messaging system. The messaging system allows for asynchronous communications between services over the ESB. In fact, when you are working with an ESB, everything is either a service (which in this context is your application software) or a message being sent between services. It’s important to note that a “service” is not automatically a web service. Other types of applications, using transports such as FTP or JMS, can also be services. Is an ESB the same thing as SOA? (Service Oriented Architecture) Not exactly. An ESB does not provide a Service Oriented Architecture, but it does provide the tools than can be used to build one–especially loose-coupling and asynchronous message passing. SOA is a series of principles, patterns, and best practices.

JBossESB [2] is an open source ESB implementation that supports multiple transports, protocols, a listener-action model for loose coupling of services, content based routing with JBoss Rules (Drools), and workflow management with jBPM.

JBoss jBPM

Business process management (BPM) is an often misused or misunderstood term.[3] There are many definitions for BPM, but the best and simplest that I can think of is that it is a systematic approach to improving business processes. What's a business process? It's important for us to understand that in this context, a "process" is not an instance of a running software program, such as the set of processes that are displayed when you type "ps" on a Unix system. In this context, a business process is the set of tasks that are performed by the systems AND the people in an organization in order to achieve that organization's goals. For example, some business processes for an insurance company would be determining risk for selling hurricane insurance in the south eastern USA, preparing actuarial tables for different demographic groups, or deciding on which pieces of junk mail to send to other demographic groups. These processes can be performed by a person, groups of people acting together, people and software services working together, or groups of services working together.

JBoss jBPM [4] is an open source business process management development environment and workflow management system. jBPM includes: a process definition language (jPDL) that combines Java and process definition, a graphical process design tool (GPD) that is implemented as an eclipse plugin, a process engine runtime (deployed to an application server) that supports long-running tasks in business processes, and a web-based console for end users to access the business process' supporting application. The GPD enables users to create process definitions by manipulating a graphic representation of the process.


A business process definition as defined in jPDL and represented in a graph in the GPD consists of multiple types of nodes that can represent tasks or a state in the process, actions which are Java code that is executed when events happen in the process, and transitions between nodes.

It's important to note the importance of state and waiting as supported by jBPM. In a long-running business process, where actions are performed by people and systems, it's often the case that the application program that supports the process has to wait for a person to do something. For example, if the process governs a company's purchasing department, the process may have to pause and wait for a person to manually approve a large purchase. Typically, when you write an application, you want function #1 to call function #2 and so on without introducing wait times. Bur, what if your application has to wait for a person to review some information and do something? We'll walk through an example of how jBPM handles waiting later on in this post. The JBossESB - jBPM integration also enables you to to "orchestrate" ESB services by invoking those services from a process definition. In this context, "orchestration" refers to combining services into a greater whole (application) while retaining some central control. Sort of like how an orchestra is comprised of many different instruments, all under the leadership of a conductor. In contrast, service "choreography" is more in the form of a collection of equal peers or partners.

In reviewing the JBossESB - jBPM integration, the three topics to consider are:

* Making Calls From the ESB to the process definition in jBPM
* Orchestrating ESB Services From jBPM to the ESB
* Exception Handling

JBossESB and jBPM Integration - Making Calls From the ESB to jBPM

Calls from the JBoss ESB into jBPM use the BpmProcessor action. This action actually uses the jBPM command API [5] to execute the calls into jBPM. Of the commands in the command API, the following (3) are available for use from ESB:

NewProcessInstanceCommand creates a new ProcessInstance using a process definition that has already been deployed to jBPM. The process instance is left in the start state so that tasks referenced by start node are executed.

StartProcessInstanceCommand is the same as NewProcessInstanceCommand, except that the process instance that is created is moved from the start position to the first node in the process graph.

As its name implies, CancelProcessInstanceCommand cancels a process instance.

All of these commands requires ESB action attributes such as process ID to be defined. The action definitions can also specify additional information in action configuration properties. For example, the "esbToBpmVars" property defines the variables that are to be extracted from the message from the ESB and set in the jBPM context. We'll look at this property when we examine a JBossESB-jPBM quickstart later in this post.

JBossESB and jBPM Integration - Orchestrating ESB Services from jBPM

Remember how we described jBPM actions as Java code that is executed when selected events happen in the business process? Well, the ESB-jBPM integration handles orchestrating ESB services from a jBPM process with the EsbActionHandler and EsbNotifier action classes. The classes are similar in that they each send a message to the target ESB service. (Don't forget that in the ESB, everything is either a message or a service.) Where they differ is that the EsbActionHandler class blocks the process and waits for a response to the message that it sends to the service. The EsbNotifier follows an asynchronous model and does not block the process and wait for a response.

How are EsbActionHandler and EsbNotifier action classes used? Since the EsbNotifier does not wait for a response to the message that it sends to a service, it should be attached to a jBPM process node's outgoing transition. This will allow the jBPM process to proceed while the message (request) sent to the target ESB service can be processed by a different thread. The EsbActionHandler, in contrast, uses a request/response asynchronous model and should be attached to a jBPM process node. When the EsbActionHandler is called, the node that is attached to waits for it to respond with a transition signal. When the node receives this signal, usually from the JBossESB callback service, the jBPM process can then continue.

JBossESB and jBPM Integration - Exception Handling - When Something Goes Wrong

A common source of problems in when you integrate products together is handling errors. In the JBossESB-jBPM integration, exceptions are handled in different ways, depending on whether a message is being passed from the ESB to jBPM, or from jBPM to the ESB.

JBossESB to jBPM exception handling is the simpler of the two. The jBPM command API (remember that the integration includes these 3 API calls: NewProcessInstanceCommand, StartProcessInstanceCommand, and CancelProcessInstanceCommand) an throw a JbpmException. The exception is not caught by the integration, but rather it goes to the ESB action pipeline, which sends the letter to the DeadLetterService and logs the message. An error message can also be sent to the ESB faultTo EPR if that property is set on the message. You define the handling of the exception by configuring an aptly-named ExceptionHandler to the jBPM process node.

jBPM to JBossESB exception handling is more complicated. If the delivery of the message to the ESB fails, then a MessageDeliveryException is thrown by the ESB ServiceInvoker. Both the EsbActionHandler and EsbNotifier actions can throw this type of exception. If the message is received by the target ESB service, but then a failure happens when it is being processed by the service, then what happens to the exception depends on whether we used an EsbActionHandler or EsbNotifier action. Remember that an EsbNotifier does not wait for a response to the message that it sends to a service. Instead, the jBPM business process continues after the message is sent to the ESB service. So, for an EsbNotifier action, the exception is not sent back to the jBPM process. If, however, an EsbActionHandler was used to send the message to the ESB service, then the jBPM business process is waiting for a response. You define the handling of the exception with a jBPM exception-transition for the EsbActionHandler.

Let's look at this integration in action by examining one of the JBossESB-jBPM quickstarts. The quickstart programs that ship with JBossESB are a great resource for learning about how to use the ESB and its integrations. At first glance, these programs may look like simple examples. But, they are much more than that. They are an extensive set of well documented models of working code that you can use as the starting point for writing your own SOA applications.

Before we examine the quickstart, we have one more setup task to perform. Now, we can run the quickstart with ESB alone installed, but anytime that you work with jBPM, you really have to look at the Graphic Process Design (GPD) tool. One of the most useful and instantly recognizable features of jBPM is how it enables you to construct a visual (graphic) representation of your business process. The process is defined in a file named "processdefinition.xml" You can always view and edit the process definition in the XML source, but its much easier to do so with the GPD.

The quickstart that we'll look at is bpm_orchestration2. This quickstart illustrates how jBPM can control a business process that executes multiple JBoss ESB processes. This quickstart makes use of synchronous calls from jBPM into the JBoss ESB processes. Let's start by looking at the quickstart's source code and its process diagram. To begin, import the quickstart as an existing project into your workspace in JBoss Developer Studio (JBDS). After the project is created, open the processdefinition.xml file to view the process design in the GPD.


One thing that you should remember when you view the process design in the GPD is that you are not looking at a static picture of the process. What you are looking at is a dynamic graphic representation of the process. Any changes that you make to the process in the GPD are reflected in the process definition source.

As I mentioned a minute ago, the bpm_orchestration2 quickstart demonstrates jBPM controlling the execution of JBossESB processes. The quickstart does this by simulating a purchase order placement and approval process. Let's examine just how this happens. To do this, we'll look at two files in the quickstart; processdefinition.xml and the JBoss ESB configuration file, jboss-esb.xml file:

jboss-esb.xml, lines 74-82:

<action name="start_a_new_process_instance"
class="org.jboss.soa.esb.services.jbpm.actions.BpmProcessor">
<property name="command" value="StartProcessInstanceCommand" />
<property name="process-definition-name" value="bpm_orchestration2Process"/>
<property name="esbToBpmVars">
<mapping esb="eVar1" bpm="counter" value="45" />
<mapping esb="BODY_CONTENT" bpm="theBody" />
</property>
</action>

The execution of this quickstart, like almost all of the JBossESB quickstarts, is started when a message is received by a listener defined in the ESB. In this quickstart, when that message is received by the listener, a jBPM process is started and moved from the start position to the first node in the process graph when the StartProcessInstanceCommand jBPM API call is executed.

At this point, let's switch over the the processdefinition.xml to see what happens in the start node. Open the processdefinition.xml file and switch to the "source" tab.

processdefinition.xml, lines 5-21
   <start-state name="start">
<transition name="" to="Receive Order"></transition>
</start-state>

<node name="Receive Order">
<action name="esbAction" class="org.jboss.soa.esb.services.jbpm.actionhandlers.EsbActionHandler">
<esbCategoryName>BPM_Orchestration2_Service1</esbCategoryName>
<esbServiceName>Service1</esbServiceName>
<bpmToEsbVars>
<mapping bpm="theBody" esb="BODY_CONTENT" />
</bpmToEsbVars>
<esbToBpmVars>
<mapping esb="BODY_CONTENT" bpm="theBody" />
</esbToBpmVars>
</action>
<transition to="Credit Check"></transition>
</node>

The start node simply transitions to the "Receive Order" node. This node defines an action using the EsbActionHandler action class. This tells us that the invocation of the action is synchronous in nature and that the jBPM process will wait for the service referenced in the action (in this case, the service named "Service1") to complete before the jBPM process continues. The rest of the quickstart continues this pattern where the jBPM process executes actions that in turn invoke JBossESB services in a synchronous manner.

At this point, let's run the quickstart and see what happens. To do this, the steps are:

Deploy the JBoss ESB archive (/build/Quickstart_bpm_orchestration2.esb) - to do this, execute "ant deploy"
Deploy the jBPM process archive (the contents of the /processDefinition directory) - to do this, execute "ant deployProcess"
Start the process - to do this, execute "ant startProcess"

After the process is started, you should see this in the server log:
20:04:36,886 INFO  [STDOUT] ** Begin Receive Order - Service 1 **
20:04:37,399 INFO [STDOUT] In: Getting Started
20:04:37,400 INFO [STDOUT] Out: Getting Started 'Receive Order'
20:04:37,400 INFO [STDOUT] ** End Receive Order - Service 1 **
20:04:38,952 INFO [STDOUT] ** Begin Credit Check - Service 3 **
20:04:38,952 INFO [STDOUT] In: Getting Started 'Receive Order'
20:04:38,953 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check'
20:04:38,953 INFO [STDOUT] ** End Credit Check - Service 3 **
20:04:40,443 INFO [STDOUT] ** Begin Validate Order - Service 2 **
20:04:40,444 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check'
20:04:40,444 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order'
20:04:40,444 INFO [STDOUT] ** End Validate Order - Service 2 **
20:04:42,704 INFO [STDOUT] ** Begin Inventory Check - Service 4 **
20:04:42,704 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check' 'Validate Order'
20:04:42,705 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check'
20:04:42,705 INFO [STDOUT] ** End Inventory Check - Service 4 **
20:04:45,093 INFO [STDOUT] ** Begin Los Angeles - Service 5 **
20:04:45,093 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check'
20:04:45,094 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check' 'Los Angeles'
20:04:45,094 INFO [STDOUT] ** End Los Angeles - Service 5 **
20:04:45,381 INFO [STDOUT] ** Begin Atlanta - Service 7 **
20:04:45,382 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check'
20:04:45,382 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check' 'Atlanta'
20:04:45,382 INFO [STDOUT] ** End Atlanta - Service 7 **
20:04:45,660 INFO [STDOUT] ** Begin Dallas - Service 6 **
20:04:45,661 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check'
20:04:45,661 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check' 'Dallas'
20:04:45,661 INFO [STDOUT] ** End Dallas - Service 6 **
20:04:49,357 INFO [STDOUT] ***** Ship It *****
20:04:49,369 INFO [STDOUT] In: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check'
20:04:49,369 INFO [STDOUT] Out: Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check' 'Shipped'
20:04:49,370 INFO [STDOUT] ***** End Ship It *****
20:04:49,392 INFO [STDOUT] SUCCESS!:
20:04:49,392 INFO [STDOUT] [Getting Started 'Receive Order' 'Credit Check' 'Validate Order' 'Inventory Check' 'Shipped' ].

Now, remember that what's happening is that the jBPM process is invoking a JBossESB service, waiting for the process to complete, and then invoking the next JBossESB services until all the services have been invoked and the jBPM process completes. Hmm. I don't know about you, but since I started working in QE, I've become skeptical of just about anything when it seems to "just work." The JBoss ESB services in this quickstart are implemented in simple groovy scripts. The scripts all run very quickly, so it's a little difficult to verify that the jBPM process is actually waiting for each service to complete before invoking the next service in the sequence defined in processDefinition.xml

So, let's make the delay to the jBPM process a little more obvious. If we add the following lines to scripts/service1.groovy and then deploy the ESB and jBPM process archive and run the quickstart again, we should see more obvious delay in the log:

println "Added new println - waiting .........."
for ( i in 0..1000 ) {
print i + " "
}
println "Added new println - done waiting .........."

To make the change effective, execute "ant undeploy" then redeploy the JBossESB archive and jBPM process archive and run the quickstart again. The server log shows:
21:45:04,966 INFO  [STDOUT] ** End Receive Order - Service 1 **
21:45:04,966 INFO [STDOUT] Added new println - waiting ..........
21:45:05,105 INFO [STDOUT] 0
21:45:05,105 INFO [STDOUT] 1
21:45:05,105 INFO [STDOUT] 2
21:45:05,106 INFO [STDOUT] 3
21:45:05,106 INFO [STDOUT] 4
.
.
.
21:45:05,312 INFO [STDOUT] 997
21:45:05,312 INFO [STDOUT] 998
21:45:05,312 INFO [STDOUT] 999
21:45:05,313 INFO [STDOUT] 1000
21:45:05,313 INFO [STDOUT] Added new println - done waiting ..........

And then, the rest of the JBossESB services are invoked in sequence, so, yes, the call to the service was synchronous and the process waited for the service to complete before it continued.

Closing Thoughts

As we discussed in the introduction to this post, one of the great strengths of the SOA Platform is the set of integrations that it supports. With its integration with jBPM you can combine a service oriented architecture implemented through the ESB with the flexibility of building applications based on your business processes designed with the jBPM graphical process designer. And, by using the JBossESB-jBPM integration, your applications can handle awkward situations where the application has to wait for a service, or a person, to perform an action. (So, unlike what Tom Petty said, waiting is not the hardest part... ;-)

References

[1] http://www.ushistory.org/franklin/quotable/quote71.htm
[2] http://www.jboss.org/jbossesb/
[3] Business Process Management definitions: http://searchcio.techtarget.com/sDefinition/0,,sid182_gci1088464,00.html
http://en.wikipedia.org/wiki/Business_process_management
[4] jBPM: http://www.jboss.org/jbossjbpm/ and the JBossESB jBPM Integration Chapter in the JBoss ESB Services Guide: http://www.jboss.org/jbossesb/docs/4.5.GA/manuals/html/ServicesGuide.html
[5] http://docs.jboss.com/jbpm/v3.2/javadoc-jpdl/

Acknowledgements

I want to thank the members of the JBossESB (see http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/Contributors.txt) and jBPM projects and the SOA Platform - especially Burr Sutter, Mark Little, Kurt Stam, and Tom Baeyans - for their help and review comments. Also, this post relies heavily on the extensive JBossESB and jBPM user documents and the quickstarts. (And, I'd also like to thank Bascha Harris for her feedback on the post!)

Monday, February 9, 2009

ESB Architecture for SOA

I've been writing a book on ESB and SOA for a while now as part of Thomas Erl's SOA series. The publishers have finally put up a page for the book. From my own perspective the content of the book is influenced heavily by what you will find in the JBoss SOA Platform and its roadmap. Of course it's not a technology specific book because it needs to be able to stand on its own, so it is no replacement for the SOA-P documentation. However, you should be able to read it and think "Ah, now that makes sense from a standards perspective", or "OK, so I'd be better of using this pattern here".

When the book is finally released we'll make sure to mention it here. In the meantime you can always go out and get another good book on enterprise SOA.

Sunday, January 18, 2009

Dog food tastes good!

We've got quite a few success stories referenced on the SOA Resource Center. There are more in the works, but if you've ever worked on success stories before you'll know that writing them up is often difficult for a number of reasons, e.g., they can't be too technical and companies are often concerned about divulging internal secrets.

Anyway that's not the point of this posting: we're working on a couple of success stories that are based on our own internal use of the SOA Platform. Of course external readers will view these as slightly less important than "real" customers, but to be honest I think that's a bad thing. If anything they show our commitment to 'put our money where our mouth is'. In each case we've taken critical systems that have been running continuously for years based on competitor technologies and replaced them with our own products. It may seem like an obvious thing to do, but you'd be surprised how many companies don't do that. The feedback we've had from those development teams has been extremely positive and hopefully I'll be able to point at the relevant success story soon. Congratulations to everyone involved!

SOA Design Patterns

I've been writing a book on SOA and ESB for a while with a couple of friends. It's due to be part of Thomas Erl's SOA series later this year. While writing it Thomas asked us to produce some patterns for his book on SOA. Well the finished book, SOA Design Patterns, is now out and it's well worth a read. A number of people throughout the industry have helped to contribute some of the patterns within it so this is definitely a cross-industry collaborative effort. Thomas also wants it to be a live-work in as much as new patterns can be contributed by anyone. So take a look at the book and if you see something missing consider contributing it. Many of the patterns are influenced by our work with the JBoss SOA Platform.