Tom Barrett's Open ESB and Mural Tutorials

Recent Additions and Updates

• Clustering GlassFish ESB: BPEL Failover and Load Balancing (V1.2) - 69 pages - October 29, 2009  (Update)
• Clustering GlassFish ESB: BPEL Failover (V1.1) - 55 pages - October 22, 2009  (Update)
• Clustering GlassFish ESB: BPEL Failover (V1.0) - 54 pages - October 1, 2009  (NEW)
• Digging Deeper into BPEL with GlassFish ESB (V1.2) - 99 pages - September 3, 2009  (Update)
• Getting Started with NetBeans (V 1.3) - 37 pages - August 3, 2009 (Update)
• Gentle Introduction to GlassFish ESB (V 1.2) - 138 pages - August 3, 2009 (Update)
• Getting Started with NetBeans (V 1.2) - 37 pages - July 24, 2009 (Update)
• Demonstrating EMail BC: Sending Out Mass Emailing (V 1.0) - 19 pages - July 10, 2009  (NEW!)
• Getting Started with NetBeans (V 1.1) - 37 pages - July 10, 2009 (Update)
• Demonstrating Yahoo! Geocoder Integration: BPEL, HTTP BC, Web Services (V 1.1) - 24 pages - June 22, 2009  (Update!)
• Event Processing with GlassFish ESB (V 1.1) - 52 pages - June 17, 2009  (Update)

Getting Started Essentials

If you are new to GlassFish ESB or OpenESB, here are my four favorite tutorials to help you get started.

Getting Started with NetBeans (V 1.3) - 37 pages - August 3, 2009

This tutorial gently introduce you to the NetBeans IDE. Open ESB, GlassFish ESB and Java CAPS 6 utilize NetBeans 6.x as the IDE. This tutorial will help you feel more comfortable with NetBeans so you can dive into Open ESB, GlassFish ESB and Java CAPS 6 ... and beyond! You'll tackle four mini-projects that come together to provide a small, HelloWorld experience with NetBeans 6. Here are the topics we'll cover:

• Creating and debugging a Java application
• Designing and testing a Java user interface
• Building an Enterprise JavaBean (EJB)
• Crafting a web service based upon an EJB
• Creating and testing a browser-based user interface to exercise the web service
  Sample Solution (zip of MyHelloNetBeans directory)

Gentle Introduction to GlassFish ESB (V 1.2) - 138 pages - August 3, 2009

The tutorial provides a comprehensive, hands-on introduction to the Open ESB technology that is making its way into Sun commercial products - GlassFish ESB and Java CAPS 6. You will tackle six mini-projects that come together to provide a small, proof-of-concept, composite application for a fictitious business called Larry's LoanMart. Here are the topics we'll explore:

• Web services based upon EJB modules and web applications
• XML schema creation and WSDL development
• BPEL orchestration of web services using the BPEL service engine
• Deployment and testing of a composite application
• Web user interface design
• Storing data in a database table via the JDBC binding component
• Reading and writing messages to a JMS queue via the JMS binding component

"Above and Beyond" opportunities help you explore:

• Using the Java Debugger
• Using the Server Log to Debug Java Code
• Exploring CASA Editor and Java EE Service Engine
• Using the BPEL Debugger
• Exploring the Ant Build Facility
• Adding Logging Statements to a BPEL Process

You might also enjoy the 50-question, self-assessment at the end of the tutorial. Answers are provided.

Solution Projects: Zip file containing all NetBeans projects at saved at the end of tutorial projects 3, 4, 5 & 6

Archive of an earlier tutorial covering similar topics.

Digging Deeper into BPEL with GlassFish ESB (V 1.2) - 99 pages - September 3, 2009

This tutorial is based upon an Internet store front portal scenario where customers purchase items.  The store front generates purchase orders (POs) forwards them to a "backend" Purchaser system. The Purchaser then collaborates with the Supplier which coordinates shipping.  When the order has been shipped, the Purchaser generates a delivery notice (DN).  The DN is picked up by Purchaser and the Purchaser matches up DNs with POs before forwarding instructions to Finance for further processing.

Specific technical topics include:

• Custom encoder to marshal XML to delimited records and unmarshal delimited records to XML
• BPEL: ForEach looping, asynchronous processing, predicates and correlation
• Service Engines: BPEL
• Binding Components: HTTP, File, JMS
• Additional Tools used: soapUI NetBeans plug-in and HermesJMS

Solution Projects: Zip file containing completed NetBeans projects for this tutorial

Archive of an earlier tutorial covering similar topics.

See: Michael Czapski's excellent screencast for the original demo upon which this tutorial is built

Event Processing with GlassFish ESB (V1.1) - 52 pages - June 17, 2009

This tutorial explores one of OpenESB's most exciting JBI components - the Intelligent Event Processor (IEP). The IEP service engine is a supported component in GlassFish ESB 2.1. It provides powerful support for event-driven architecture (EDA), in general, and complex event processing (CEP) and event stream processing, (ESP) in particular. EDA is a software architectural pattern focusing on producing, detecting, consuming and acting upon events. An event is a significant change in state of something of interest to the business.

At Java One 2008, Sun showed the PennyLotto demonstration in the vendor pavilion area. The demo introduces how Open ESB technology supports complex event processing (CEP). The beauty of the demo is that it is simple and there is no elaborate scenario or sophisticated business policy to get in the way of understanding how the NetBeans tooling and the IEP provide EDA facilities. I thought highly enough of the demo that I have documented it here in tutorial form.

Topics include:

• Creating an event processor that utilizes these operators: input stream, time-based window, event correlator/filter, output stream and a save stream output
• Assembling and testing event processor as a composite application
• Storing KPI data
• Exploring "behind the scenes" to see how IEP stores data as it flows through an event model

Sample Solution (zip of MyIEPTutorial directory)

Archive of an earlier tutorial covering similar topics.

Clustering GlassFish ESB: BPEL Failover and Load Balancing (V1.2) - 69 pages - October 29, 2009

After reading the available documents on GlassFish ESB BPEL failover available on the web, I craved a single, end-to-end tutorial that would handhold me through the steps of establishing a cluster of application server instances, setting up BPEL service engines for persistence and proving that BPEL failover really works in a cluster. During the step-by-step treatment, I would be looking for all the whys and hows that naturally come to mind when learning something new that is reasonably complex. So, this tutorial hopes to address the instructional needs I craved.

We'll setup a very simple failover situation so you can experience BPEL failover in a GlassFish cluster. We'll create a cluster of two application server instances each hosting a BPEL service engine (SE) and two binding components (BC), HTTP and FILE. We'll feed test messages to both BPEL processes running in each BPEL SE. The test messages will be sent directly to the HTTP BC on each instance. We are not using any load balancer in this scenario. Once the messages have all arrived and business process instances have been created, we'll kill one of the application server instances and watch as the BPEL SE on the surviving instance picks up the in-flight work and completes all business process instances.

After we have experienced BPEL SE failover, we'll turn our attention to configuring and demonstrating how load balancing could be added to the scenario. In our failover testing, we sent SOAP requests directly to the two HTTP BCs. However, in a “real world” solution, you would instead add a load balancer to the infrastructure and send requests to it.

Topics include:

• Building and configuring a GlassFish cluster of two application server instances
• Creating and deploying a simple BPEL process that we can use for BPEL failover testing
• Establishing the soapUI test cases we need to demonstrate BPEL failover
• Seeing BPEL failover in action
• Installing, configuring and testing the Sun Web Server load balancing plug-in to front our GlassFish ESB cluster

Additional Tutorials

Exploring Sun ESB Suite: Open ESB Technology (V 1.1) - 51 pages - November 18, 2008

This tutorial explores a health care scenario that analyzes HL7 ADT (Admission Discharge Transfer) records to identity patient length of stays that have exceeded an average threshold set by management.  The Intelligent Event Processor (IEP) computes a moving average and identifies ADT records that note an exceptional patient length of stay.  IEP generates an alert for each excessive length of stay detected.

Specific technical topics include:

• Intelligent Event Processor (IEP) (Time-Based Window, Relation Aggregator, Correlator / Filter)
• BPEL orchestration
• File BC for ADT record input
• BPEL process invokes IEP via HTTP binding component
• IEP uses File BC for output
• Service Engines: BPEL, IEP
• Binding Components: HTTP, File

See: Michael Czapski's excellent screen cast for this demo

Exploring Sun MDM Suite: Open ESB and Mural Technology (V 1.0) - 111 pages - November 6, 2008

This tutorial explores a health care scenario where multiple patient master databases exist containing duplicate and inconsistent patient information. A patient master index is defined and a patient application is generated that stores its master index in MySQL, provides a web-based interface to maintain index records, surfaces web services that can be called by other applications to do programmatic access to the index records and supports a JMS topic to broadcast master index changes to other hospital applications.

Specific technical topics include:

• Defining patient data model
• Establishing MySQL tables
• BPEL process feeding index via web service call
• BPEL process "listening" on JMS topic for index updates
• Using Master Index Data Manager web app
• Service Engines: BPEL
• Binding Components: File, JMS

See: Michael Czapski's excellent screen cast for this demo

Errata:  If you have trouble with this version of the tutorial, please see these errata notes.  Thanks to all who have provided feedback. I'll address these issues/comments in the next version of the tutorial.

SOA Pipes and Filters: Open ESB Technology (V1.5) - 37 pages - December 30, 2008

Most of us are familiar with the notion of pipes and filters. I have found that they can provide an instructive way of looking at building composite applications with SOA technologies like Open ESB.   Software pipelining is the linking together of a series of independent processes (filters) chained together by anonymous pipes (|). Output (stdout) from a process provides input (stdin) to the next one. The business integration patterns community, led by Gregor Hohpe and Bobby Woolf in their seminal book ("Enterprise Integration Patterns"), appreciate the notion of software pipelining.

In this tutorial, we'll create two simple web services that provide several operations (filters). The first web service is an incredibly simplistic encrypter that takes an input string and reverses it, flips pairs of characters or applies the ROT13 algorithm. The second web service is a formatter and transforms input into three output styles: all upper case, all lower case or as an HTML page. So, in our software pipeline, we'll have six filters to play with: reverse, flipPairs, rot13, upperCase, lowerCase and htmlize.  A BPEL process will play the role of the UNIX shell to orchestrate the sequential execution of the our encryption and formatting filters. The end result is a not-so-super-secret encrypter/decrypter with sophistication that might appeal to a fifth-grader interested in cryptograpy!

Exploring JCA Message-Driven Beans: Integrating with "Foreign" JMS Brokers (V1.0) - 55 pages - January 5, 2009

JMS is a standardized protocol, but the JMS specification leaves room for message broker vendors to include "secret sauce" to create reliability and performance differentiators. As a result, JMS-based message brokers from two vendors may not be interoperable. The JMSJCA library addresses this as it abstracts away the differences between JMS servers.

In the Open ESB community, Sun has led a JMSJCA project (https://jmsjca.dev.java.net/) to create a JMSJCA library that value-adds to the basic JCA framework to host an interface into JMS message brokers. In other words, the JMSJCA library provides a mediation layer between the EJB container hosting the MDM and the message broker hosting the queues and topics. This helps solve a nagging problem enterprises face when using JMS implementations from multiple vendors.

In this tutorial, we'll build a simple JCA MDB that reads messages from a JMS queue and then transfers them to a topic. We'll test the MDB using GlassFish's embedded Java MQ message broker. Then, to prove the interoperability that the JMSJCA library provides for JCA MDBs, we'll install JBoss, configure it and (without any MDB code changes) we'll exercise our MDB using a JBoss queue and topic.

Managing & Monitoring Java CAPS 6: Open ESB Technology (V1.1) - 62 pages - September 5, 2008

The tutorial explores each of four layers of monitoring and management:

• JMX
• Java Managment & Monitoring API
• Command Line Interfaces (asadmin and asant)
• Graphical User Interfaces (CAM and GlassFish Admin Console)

The tutorial contains four mini-projects, one for each layer.  We'll start at the lowest levels of abstraction (JMX MBeans) and work our way up to the highest level (graphical user interfaces).  Although the GUIs provided by the Composite Application Manager (CAM) and the GlassFish Console offer more "sizzle," I think you will appreciate the bottom up approach.  It will help you understand what underlies the GUI functionality and you will be aware of alternative approaches if the GUIs fall short of your needs.  For example, having JMX, Java API and CLI alternatives can be quite handy for developing your own scripting utilities.

BAM Dashboards for Open ESB and Java CAPS (V1.0) - 42 pages - September 29, 2008

This tutorial introduces you to building a BAM (business activity monitoring) dashboard display using the NetBeans IDE. You'll use JasperSoft's iReport plug-in module for NetBeans. iReport is an open source offering that allows you to create sophisticated reports for display in either rich clients or in browsers. In his tutorial, you'll use iReport's design facility to create and test a report specification. Then, you'll explore iReport's design/compile/fill/export life cycle as you develop a web application that reads database data generated by the Java One 2008 PennyLotto IEP demo and constantly refreshes a pie chart summary. Here are the four projects:

• Installing the iReport NetBeans Plug-in
• Establishing a Test Database
• Building a Report with iReport
• Developing the Auto-Refreshing Dashboard

Errata:  If you have trouble with this version of the tutorial, please see my errata notes.  On 12/19/2008, I quickly worked through the tutorial with latest software from Open ESB and JasperSoft and noted changes in this errata document that I will be making as I next revise this tutorial.

Demonstration Writeups

Demonstrating Yahoo! Geocoder Integration: BPEL, HTTP BC, Web Services (V1.2) - 24 pages - June 22, 2009

This demonstration walk through is based on a scenario that you have database records that contain address information and you want to geocode them by capturing longitude and latitude for each. Yahoo! Maps Web Services provides a Geocoding API. This RESTful API allows you to programmatically lookup latitude and longitude for any address. Your application can extract the latitude and longitude and store it along with the database address record or use it to plot data points in a mashup, for example. The task in this demonstration is to show how to use NetBeans tooling and Open ESB-based technology to interface with the Yahoo! service.

We explore two approaches. Both approaches create a SOAP web service that wraps the HTTP interaction to the RESTful Yahoo! web service. This shields your SOAP-focused, SOA developers from having to deal with both REST. We can foster an "all SOAP shop." The SOAP web service we create here is simply a facade to hide the RESTful interaction. Here are the two approaches:

• Approach 1 (Using JBI: BPEL SE and HTTP BC) creates a solution featuring JBI. The BPEL service engine hosts the logic and the HTTP binding component (BC) provides connectivity to Yahoo!. This approach is light on hardcore programming in favor of using the BPEL, WSDL and XSD editors in the NetBeans toolset. "Near zero coding?"
• Approach 2 (Using a EJB-Based Web Service) develops a web service implemented as as a stateless EJB. This approach is more of a traditional Java EE and web services coding exercise. There's no JBI here.

At the end, I have some comments on the pros and cons of each approach.
Sample Solution (zip of GeocoderBPELDemo directory)

Demonstrating Database Operations with the Database BC (V1.0) - 45 pages - January 22, 2009

This demonstration walk through is based on a scenario where a production database exists on a remote server. You want to show a database select that joins over two tables. The result set from the select join is to be used to populate an extract table that exists on your local database. So, we want to grab the records we want from the remote, production database and insert them into a local Java DB database for convenient development and testing.

The demo features the Database Binding Component to:

• Perform the select join on the remote database (the built-in "samples" Java DB database)
• Clear the local, extract table before inserting the latest extracted records
• Insert extracted records into the extract table on the local database (new Java DB database called "extract" that we will create)

The scenario gives us a chance to show the Database BC performing a select join across two tables via a prepared statement. In addition, we'll use the Database BC inside a BPEL while loop to insert the selected records into the extract table. Finally, Database BC will delete records in the extract table before we insert a new batch.

We'll build two BPEL processes. One will be a "sub process" as it is called by the other. We'll utilize these JBI components:

• BPEL SE (one BPEL process calling another BPEL process)
• SOAP BC (simple RPC literal, request/response message exchange)
• File BC (write operation overwriting previous contents of the output file)
• Database BC (select (prepared statement), insert and delete operations)

Demonstrating EMail BC: Sending Out Mass Emailing (V1.0) - 19 pages - July 10, 2009

Lets assume that you want to see the EMail binding component in action. You are looking for a Hello World sort of application where the business scenario doesn't get in the way of understanding the technical features and the steps required to use them. However, the Hello World must be "interesting."  You think you have found a candidate personal application.

In order to save money, you canceled the weekly pool service contract and are counting on your three sons to adjust the chemicals and remove leaves from the bottom of the pool each week. In exchange for performing the weekly pool maintenance chores for free, you agree to continue to feed, clothe and
house them. It sounds like a pretty good deal for them, but you are constantly having to remind them.  You would like to be able to easily ding them with an email as a reminder. Sending emails from a BPEL process that utilizes the EMail Binding component makes for a good Hello World candidate.

The application is to be very simple.  It is to take in a list from a file of one or more email addresses and send out email reminders to each email address in the input file.  For example, you can load up the input file with the email addresses of your three forgetful son's and generate a pool maintenance reminder email for each address.  You hope that this will enhance the probability that one of your sons will tackle the pool chores. You envision firing this email generation process by sending a SOAP message to a web service endpoint.

You build two BPEL processes. The second one invokes the first one:

• BPEL Process #1: Sending the Email
• BPEL Process #2: Sending Multiple Emails from Input File Data

MDM Details: Sun Partner MDM Training

The following four tutorials were developed to help Sun partners have a successful hands-on introduction to the Sun MDM (Master Data Management) product based upon Mural open source technology.  The tutorial flow and steps were based upon a Sun internal technical deep dive session done by Sun engineering in July 2008 in Monrovia, California. 

1. Building a Master Index (V 0.1) - 22 pages - September 16 2008

Focus is on the initial configuration and creation of a master index.  Topics include:

• Create Master Index Application
• Configure Matching Parameters
• Tailor Standardization Support
• Specify Record Blocking
• Establish Filters
• Build Master Index Application
• Build Master Index Database
• Configure Application Server
• Deploy and Test Master Index Application

2. Customizing MIDM (V 0.1) - 12 pages - September 16 2008

MIDM is the Master Index Data Manager web application that is produced to allow humans to perform manual maintenance and searching of the master index.  It is a highly configurable web application.  Here are the customization topics addressed:

• Change Landing Page
• Change Tab Titles and Ordering
• Add New Search Screen
• Modify Search Result Summary Screen
• Create Master Index Application
• Configure Matching Parameters

3. Preparing Data and Loading (V 0.1) - 36 pages - September 16 2008

To initially populate a newly created master index, a project team goes through a process of profiling, cleansing, batch matching and bulk loading. Once the index has been populated, humans can use the MIDM web application to maintain and search the index and applications can bi-directionally integrate with the master index programmatically. Here are the data preparation and loading topics addressed in this tutorial:

• Build Master Index Application
• Extract Source Data into Staging
• Profile and Cleanse Staged Data
• Bulk Match Staged Data
• Bulk Load Master Index Database
• Browse Master Index Using MIDM

4. Integrating with ESB (V 0.1) - 36 pages - September 29 2008

Once the master index is built and populated, it can be programmatically maintained and searched by using the ESB technology that is a key part of the Open ESB-based offering. In this tutorial, we'll create applications that 1) subscribe to master index changes to feed down stream system and 2) access the client from a Java client. Here are specific topics address:

• Subscribe to Master Index Changes
• Exercise Master Index from a BPEL Process
• Call Master Index from Java Client
• Performing a Search from a Web Application