Jonas Partner 2010-02-24T17:23:19Z WordPress http://www.jonaspartner.com/?feed=atom Jonas http:// <![CDATA[Spring Integration: Whats going on in there? Part 1 Esper wire tap]]> http://www.jonaspartner.com/?p=335 2010-02-24T17:23:19Z 2010-02-24T16:27:08Z Earlier this week Russ Miles announced the release of the OpenCredo Esper extension project (read it here).  This project is something we created primarily to make using Esper, the open source Complex Event Processing framework, in conjunction with Spring Integration really easy.  At OpenCredo we have been using Esper to help solve the problem of comprehending and monitoring the state of messaging applications for a while now.  Understanding the state of an application is one of the issues we repeatedly see with clients looking to adopt an Event Driven Architecture based on Spring Integration.  Esper allows us to create views of the data as it passes through our asynchronous event based application which helps to eliminate some of the complexity created by breaking our application into a series of loosely coupled components.

In the first post on this topic I will be walking you through a simple example that takes an existing Spring Integration application and shows how to use the Esper wire tap to send messages passing over a Spring Integration channel into Esper. 

In the next post in the series I will show how you can hook Spring Integration into Esper to have events trigger the creation of Spring Integration messages.

Sample Application

Our sample application consists of a Spring Integration message pipeline that is connected to a HR system.  Whenever the HR process successfully hires a new employee at our fictitious company the HR system creates a notification message that allows other systems to take appropriate actions to deal with the new hire.

To hide the fact that we are using Spring Integration from the HR system we use a Spring Integration gateway which is injected into the HR system. The SI gateway creates a proxy implementation of the configured interface, shown below, which is then injected into HR System.

public interface EmployeeConfirmationNotifier {

  public void employementConfirmed(EmployeeConfirmation confirmation);

}

For testing purposes we expose the HR system as a JMX bean with the two methods we use to instruct HR to hire new developers or lawyers exposed as managed operations.

@ManagedResource
public class HrSystem {
  private final AtomicLong employeeNumber = new AtomicLong(0);
  private final EmployeeConfirmationNotifier confirmationNotifier;

  public HrSystem(EmployeeConfirmationNotifier confirmationNotifier) {
    this.confirmationNotifier = confirmationNotifier;
   }

  @ManagedOperation
  public void addMoreLawyers(int count){
    for(int i = 0 ; i &lt; count ; i++){
      //do some hiring
      this.confirmationNotifier
               .employementConfirmed(
                        new EmployeeConfirmation(
                          this.employeeNumber.getAndIncrement(), "Lawyer"));
    }
  }

@ManagedOperation
public void addMoreDevelopers(int count){
for(int i = 0 ; i &lt; count ; i++){
      //do some hiring
      this.confirmationNotifier
               .employementConfirmed(
                        new EmployeeConfirmation(
                          this.employeeNumber.getAndIncrement(), "Developer"));
    }
  }

}

Then simply add the MBean export to the configuration file.

  <context:mbean-export />

Adding in the wire tap

The OpenCredo Esper Extension project provides a namespace for configuring the wire tap. All that is required is to use the wire-tap element from the si-esper namespace. An example of the esper wire tap is shown below:

<si-esper:wire-tap id="defaultWireTap"
       template-ref="defaultEsperTemplate" sourceId="hrSystem"
       post-send="true" send-context="false"/>
   
<si-esper:wire-tap-channels default-wire-tap="defaultWireTap">
  <si-esper:channel pattern="confirmedEmployeesChannel" />
</si-esper:wire-tap-channels>

<esper:template id="defaultEsperTemplate">
  <esper:statements />
</esper:template>

Here we have configured a wire tap to publish an event to Esper each time a message is sent to the confirmedEmployeesChannel channel. By setting send-context to false the published event will simply be the payload of the Spring Integration message, in this case an instance of EmployeeConfirmation. If send-context is set to true, the default, then additional context such as the channel and the sourceId will be included in the event published to Esper.

That’s all that is required to make the data passing available through Spring Integration channels available to Esper.

Next time

In the next post I will show how to then use Esper to query that flow of data allowing us to better understand what is happening at runtime and to alert us to interesting conditions in our application. The next post will also include the runnable source code for the example.

]]> 0 Jonas http:// <![CDATA[Quiet but not forgotten]]> http://www.jonaspartner.com/?p=312 2009-10-25T13:41:10Z 2009-10-25T13:36:16Z Spring Integration in Action

Little time to blog lately but hopefully I will be getting some more time as of mid November.  Having started OpenCredo in conjunction with Russ Miles and Gary Levy  earlier in the year we have seen fantastic growth in our first six months.  Thankfully things are now starting to settle down after the initial crazy days.  Over the next few months we are expecting to find time to deliver some interesting open source projects which combine some of our main company interests of integration, messaging and cloud computing.  Watch this space.

In addition I have been working on the upcoming Spring Integration in Action with the other Spring Integration committers.  A first view of the book is now available through the Manning early access program here.  Feedback through the Manning forum would be much appreciated.

]]> 1 Jonas http:// <![CDATA[Working with Maven and SpringSource dm Server]]> http://www.jonaspartner.com/?p=185 2009-03-09T19:01:58Z 2009-03-08T22:07:16Z Since I am a big Maven fan one of the things that pained me when starting to work with the dm Server was that the tooling for managing project dependencies was all based around Eclipse.  The STS 2.0 RC discussed by Christian Dupuis here now has support for a separate test classpath using a proprietary TEST.MF.  To me this still poses a problem when you start thinking about Continous Integration testing.  Currently there is going to be duplication and you will need to keep the classpath understood by your build system and your IDE in synch manually. It is however surprisingly easy to setup Maven and dm Server to play nice, so before posting the code for my recent Spring Integration with dm Server demo from the UK Spring User Group I thought I would walk through setting up dm Server with Maven.

The first thing you need to decide is where you are going to have your shared repository which will be populated by Maven and used by dm Server to resolve dependencies.  You can configure dm Server to use the default Maven repository under your home directory, however for reasons of clarity I prefer to maintain  a separate repository.  So, assuming you have downloaded dm Server and unzipped it, the first thing you will need to do is modify the provisioning configuration in DM_SERVER/config/server.config. By default there is no section for provisioning here with convention being used, so after the osgiConsole section define a section as below.

"osgiConsole": {
    "enabled": true,
    "port": 2401
},
"provisioning" : {
    "searchPaths": [
        "repository/bundles/subsystems/{name}/{bundle}.jar",
        "repository/bundles/ext/{bundle}",
        "${user.home}/servers/dm-server-maven-repo/**/{bundle}.jar",
        "repository/libraries/ext/{library}",
        "repository/libraries/usr/{library}"
    ]
}
}

To test this we will create a simple project which uses Apache Commons lang StrSubstitor class to print the simple message given as an example in the Javadoc for commons lang “You are running with java.version = ${java.version} and os.name = ${os.name}.”)

To create the sample Maven project we can use the Maven archetype plugin so go to the command prompt in the directory where you want to create the test project.

  1. At the command prompt “mvn archetype:generate”
  2. Then select the number relating to maven-archetype-quickstart , 15 by default but may vary according to your setup
  3. Enter a groupId “org.jpartner” in my case
  4. Enter an artifactId “mavenDmHello”
  5. Hit enter to accept the default version “1.0-SNAPSHOT”
  6. For package I used the same value as groupId so “org.jpartner” in my case

Ok, so now we need to open up the pom.xml that has been created for use by the archetype plugin, so change directory to mavenDmHello and open pom.xml in your preferred text editor. After the URL element and before the dependency element add in the below xml fragment to allow Maven to resolve dependencies from the SpringSource Enterprise Bundle Repository, which you can browse here http://www.springsource.com/repository.

<repositories>
    <repository>
        <id>com.springsource.repository.bundles.release</id>
        <name>SpringSource Bundle Releases</name>
        <url>http://repository.springsource.com/maven/bundles/release</url>
    </repository>
    <repository>  <id>com.springsource.repository.bundles.external</id>
        <name>SpringSource External Bundle Releases</name>
        <url>http://repository.springsource.com/maven/bundles/external</url>
    </repository>
</repositories>

To declare the dependency on Apache Commons Lang we are adding the version from the SpringSource repository to ensure we get an OSGi compatible jar, hence the non-standard artifactId.

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>com.springsource.org.apache.commons.lang</artifactId>
     <version>2.4.0</version>
</dependency>

In order to tell Maven to use the non-standard local repository location we could configure the location in our global settings.xml. However, since this will effect all our maven projects, for the moment we will set up a separate settings.xml file. In the root of the created project create a file called settings.xml with the content below.

<settings>
   <localRepository>${user.home}/servers/dm-server-maven-repo</localRepository>
</settings>

To tell Maven about our new settings.xml we pass the -s so running mvn -s./settings.xml install will download our dependency on Commons Lang into our new repository. Maven will of course download a whole host of other things as well.

Next we will create a class in our project which makes use of StrSubstitor to make sure everything is working as expected. So create a greeter class as below.

package org.jpartner;

import org.apache.commons.lang.text.StrSubstitutor;

public class Greeter{
   public Greeter(){
      String replaced =
         StrSubstitutor.replaceSystemProperties(
            "You are running with java.version" +
                " = ${java.version} and os.name = ${os.name}.");
      System.out.println(replaced);
   }
   
}

We can then use the standard mechanism to have an instance of our Greeter class created at runtime. So under src/main/resources create a directory structure META-INF/spring and inside the spring directory create a file called greeter.xml as below.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation=
  "http://www.springframework.org/schema/beans
     http://www.springframework.org/schema/beans/spring-beans.xsd">

   <bean id="greeter" class="org.jpartner.Greeter" />

</beans>

The next thing we need to do is provide an OSGi compatible manifest importing the org.apache.commons.lang.text.StrSubstitutor. Because Maven will auto generate a manifest we also need to configure the maven-jar-plugin to use our provided manifest. We do this by adding the build section as below to our pom.xml.

<build>
   <plugins>
      <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-jar-plugin</artifactId>
         <configuration>
         <archive>
            <manifestFile>
               ${basedir}/src/main/resources/META-INF/MANIFEST.MF
            </manifestFile>
         </archive>
         </configuration>
      </plugin>
   </plugins>
</build>

Then create the MANIFEST.MF in the META-INF directory as below. We declare our import of the commons lang package containing the StrSubstitutor

Manifest-Version: 1.0
Bundle-Name: Service_interfaces Bundle
Bundle-ManifestVersion: 2
Bundle-SymbolicName: mavenDmHello
Bundle-Version: 1.0.0
Import-Package: org.apache.commons.lang.text;version="[2.4.0,3.0.0)"

Now start up the dm Server using the script under DM_SERVER/bin. Run mvn install again to create a jar containing our new manifest. Then simply copy the jar created in the maven target directory to the DM_SERVER/pickup. You should see dm Server output as below.


[2009-03-08 20:39:22.735] fs-watcher Processing ‘INITIAL’ event for file ‘mavenDmHello-1.0-SNAPSHOT.jar’.
[2009-03-08 20:39:23.504] fs-watcher Deployment of ‘mavenDmHello’ version ‘1′ completed.

A tail of the serviceability log DM_SERVER//serviceability/trace/mavenDmHello-1/trace.log will show something similar to below, showing that we are successfully using Maven to populate the repository for use with dm Server.


[2009-03-08 20:39:23.482] server-dm-4 amework.beans.factory.support.DefaultListableBeanFactory.unknown I Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@65d0e7e9: defining beans [greeter]; root of factory hierarchy
[2009-03-08 20:39:23.493] server-dm-4 System.out I You are running with java.version = 1.6.0_07 and os.name = Mac OS X.

So now we have dependencies working with Maven however we are still creating the manifest by hand. The SpringSource Maven bundlor plugin can be used to help with that and I will be posting an example using that soon.

]]> 0 Jonas http:// <![CDATA[Demo One from my UK Spring User Group talk: The simplicity of Spring Integration]]> http://www.jonaspartner.com/?p=28 2009-02-24T10:06:58Z 2009-02-22T18:06:57Z I was asked to make the code available for the demos that were part of my talk for the UK Spring User Group at Skills Matter. The video of which is here.  Today I am posting the code for the first demo which demonstrates just how easy it is to get up and running with Spring Integration.  The demo shows a simple application which takes requests from JMS, routes according to business logic and calls through to a Java service to process the message. The projects used in this demo can be downloaded here febSugDemo1.zip.  The zip contains three Eclipse projects along with Maven poms so Eclipse is not a must.

  1. Application Generator: This project is there for testing and when run will create random LoanApplication instances and send them for processing over JMS. This can be started using LoanApplicationGenerator in src/main/java
  2. Demo 1 Quotes:  Contains the Spring Integration configuration and simple service implementations used to process applications.  Can be started from LoanQuoteStarter under src/test/java
  3. Loan Quote Domain: The LoanApplication and Quote types used by both previous proejcts

The example is that of a simple message flow which first receives a LoanApplication from a JMS queue.  The loan application is then routed based on simple business rules to either a quote generating service or to a rejection queue for processing later. I will start off with a detailed walk through of the example application so if you just want to get the code up and running you can skip through to the getting it running section towards the bottom.

Loan application message flow

loanquoteflow

Walk through

The configuration for the demo1 project can be found under src/main/resources.  The loanquote-context.xml first defines an inbound JMS channel adapter.  The fact that we are using a JMS channel adapter rather than a JMS gateway indicates that we are doing one way communication.  If we were participating in a request response scenario we would want to use one of the provided gateways instead.  The inbound adapter means we are receiving from JMS and placing an instance of  org.springframework.integration.core.Message on the configured channel.  Spring Integration also provides an outbound gateway to take a Spring Integration message from a Spring Integration channel and send that to a JMS destination.  JMS gateways also come in both inbound and outbound versions.

JMS Inbound channel adapter

    <jms:inbound-channel-adapter
        connection-factory="connectionFactory"
        destination-name="newApplications"
        channel="requests">
</jms:inbound-channel-adapter>

<channel id="requests" />

The JMS inbound adapter is configured very simply in this case with a reference to a JMS connection factory, the name of a JMS destination and the name of the Spring Integration channel on which received messages will be placed. Our connection factory is a Spring bean defined elsewhere.  In this case we are using the ActiveMQ configuration below.

ActiveMQ connection factory

<beans:bean id="connectionFactory"
    class="org.apache.activemq.spring.ActiveMQConnectionFactory">
    <beans:property name="brokerURL"
        value="tcp://localhost:61616" />
</beans:bean>

The next step in our flow is to route according to the credit rating which is already available in the LoanApplication. To do that we use a simple router implementation as below. Note that rather than accepting an instance of org.springframework.integration.core.Message the router accepts an instance of LoanApplication which is the payload of the message. This makes the router easier to unit test by decoupling it from messaging. The router in this case is simply returning a String channel name which will then be resolved to a channel instance by the framework. The default strategy, which will be used here, is to look for a Spring bean in the application context with an id matching the String returned by the router. This strategy is as with most things in Spring Integration pluggable if required.

Credit Checking Router

public class CreditCheckingRouter {
   
    public String route(LoanApplication loanApplication){  
        if(loanApplication.getApplicant()
                .creditBetterThan(CreditRating.AVERAGE)){
            return "quote";
        }
        return "reject";
    }
   
}

In order to connect the router into our integration application we simply declare the POJO router as a Spring bean and reference this from our router element. Our router element requires a reference to the channel from which messages will be received, a reference to the POJO router bean and the name of the method to invoke when a message is received. We also need to declare the two channels which our router will route to. Here the quote channel will be of the default DirectChannel type. This means that a call to send on this channel will dispatch the message to any consumer(s). In the case of rejected applications we are going to queue those up and process them later so we declare a queue element as a child of channel. This means that the reject channel will be an instance of QueueChannel and calls to send will not result in any message dispatch. The consumer responsible for processing rejections will need to come looking for our rejected loan applications by polling the queue.

Router configuration

<beans:bean id="creditCheckRouter"
    class="org.jpartner.CreditCheckingRouter" />

<router input-channel="requests" ref="creditCheckRouter"
    method="route" />

<channel id="quote" />
<channel id="reject">
    <queue />
</channel>

Our quote service is very simple in this case. If you have the good or great credit rating to get past our router then we are simply going to quote our standard fixed rate of 20% APR, sign of the times I’m afraid. Our service activator is again very testable and decoupled from the messaging framework, the LoanApplication is the payload of the message received from the input channel and the returned Quote becomes the payload of the message sent to our output channel.

Quote Service

public class QuoteService {
   
    public Quote createQuote(LoanApplication loanApplication){
        return new Quote(20,loanApplication);
    }

}

Hooking our quote service in is then just a case of declaring a service activator with a ref to the bean, the input and output channels. For the purposes of the demo we are then simply printing our quotes to standard out using a provided stream channel adapter.

Loan Quote Service Activator

<beans:bean id="quoteService" class="org.jpartner.QuoteService" />
<service-activator input-channel="quote"
    output-channel="createdQuotes" ref="quoteService"
    method="createQuote" />

<channel id="createdQuotes" />
<stream:stdout-channel-adapter id="stdout"
    channel="createdQuotes" append-newline="true"/>

IDE Setup

Since the projects are using Maven, if you don’t have a Maven plugin already installed in Eclipse I would recommend installing Q4E from here.
You can then import the projects from the zip using File-> Import -> General -> Import existing projects into workspace and navigate to the archive.

Running it

We sometimes get questions on the Spring Integration forum about starting up and deploying a Spring Integration application. The short answer is that all you need is a JRE, since Spring Integration is designed to be lightweight and it does not require an application server to run. In the case of the demo the application is started using a main method. In this case however since we are using JMS you will need a broker. I used ActiveMQ 5.2 which you can download here. Simply unzip it and run activemq-admin start from the bin directory. You should then be able to browse to http://localhost:8161/admin/queueGraph.jsp and see a number of example destinations. Start the LoanApplicationGenerator by running the main method. So Run As-> Java Application if you are using Eclipse. Refreshing the ActiveMQ console should now show you that we have a build up of loan applications on our newApplication JMS queue. To start our loan quote application simply run the LoanQuoteStarter class in the demo project, the code of which is shown below. This creates our Spring application context and since our beans are lifecycle aware the application will immediately start processing loan applications.

Startup class for loan processing application

public class LoanQuoteStarter {
   
    public static void main(String[] args){
        new ClassPathXmlApplicationContext("loanquote-context.xml",
            LoanQuoteStarter.class);
    }

}

You should now see console output as below.

Applicant name:Sue rating: GOOD amount:79053.0 years:8 quoted rate : 20.0
Applicant name:Greg rating: GOOD amount:47561.0 years:1 quoted rate : 20.0
Applicant name:Simon rating: GREAT amount:41472.0 years:7 quoted rate : 20.0
Applicant name:John rating: GOOD amount:19438.0 years:1 quoted rate : 20.0
Applicant name:Emma rating: GREAT amount:66612.0 years:3 quoted rate : 20.0
Applicant name:Bob rating: GOOD amount:80851.0 years:0 quoted rate : 20.0
Applicant name:Greg rating: GOOD amount:19175.0 years:9 quoted rate : 20.0
Applicant name:John rating: GOOD amount:18578.0 years:2 quoted rate : 20.0
Applicant name:Simon rating: GOOD amount:64692.0 years:4 quoted rate : 20.0

The next post will be based on demo number two from my talk which will show how we can break up an application such as the example loan quote application into OSGi bundles, and in doing so we will create a more modular, testable and flexible application.

]]> 8 Jonas http:// <![CDATA[Welcome to my new online home]]> http://www.jonaspartner.com/?p=12 2009-03-08T22:27:09Z 2009-02-21T04:00:07Z Welcome to my new online home. I am currently in the process of establishing a new company with a number of like minded individuals but until we have the new site up and running I have decided to resurrect www.jonaspartner.com. The new company will be offering consultancy and development services primarily around Open Source whilst at the same time expending a lot of our time contributing to Open Source projects which we think are exciting.  We will be looking to pioneer some new ways of collaborating with clients to drastically reduce the risks, both technical and financial, associated with many IT projects.  We see this as the key to persuading people that technology projects can be a good investment in the current times. More on that soon.

Over the next few months I will also be putting out a number of blogs on topics I have been meaning to write something on for some time. Most of these will probably relate to Spring Integration since that is definitely what I am most excited about right now although the ability of Grails to make web development with Java exciting and fast paced is also up there for me.

Certainly the first few blogs will be follow-ups to the well received London Spring User Group talk on Spring Integration I delivered on the 12th of February at Skills Matter. The video of that talk can be found on the Skills Matter website here. Thanks to Skills Matter for hosting yet another great community event and to Cake Solutions for providing the organisation and the generous post-talk beer for everyone at the Crown. In the next few days I will be putting up the code from the samples in that talk along with a walk through of the unsuccessful Spring Integration with OSGi demo running on the SpringSource dm Server.

]]> 0