Loosely coupled systems

When multiple processes or modules interact with each other within a large enterprise application, the application is considered to be robust and scalable if the individual modules are loosely coupled. When the modules are loosely coupled, each of them can grow or change independent of the other as long as the external contracts or interfaces remain same.

A proven pattern to ensure that the systems are decoupled is to follow a message oriented architecture where the message schema will constitute as the layer of separation. When multiple processes are present within an enterprise application, it is a bad idea for them to be tightly integrated with each other by knowing the internal implementation details of one another beyond just the message schema and version used at the layer of separation.

A recommended approach is to use an EAI pattern such as Publisher-Subscriber (or simply pub-sub) approach to decouple the different processes. This approach is the underlying principle when designing an enterprise service bus. We will look at designing and implementing such a bus in the following sections. The example illustrated below portrays a high level vision for a sample enterprise application.

Let us examine the diagram in details.

• There are processes P1, P2...P8 which follow message oriented architecture. They share service contracts and schemas, not classes.
• There is a "bus" which is the mechanism by which the messages are sent to the right destination once it is sent by a sender. We will break this logical diagram into a detailed implementation using MSMQ in the subsequent section.

In this example the multiple processes which work with each other. Since they use the pub-sub pattern for message communication, they do not have any knowledge of presence of each other. This is because there is no direct link between any of the processes. All the messages are sent and retrieved from the "Enterprise Service Bus". The ESB is realized by the underlying MSMQ infrastructure in this application.

Figure 1: Logical diagram of an Enterprise Architecture

Infrastructure

For the pub sub infrastructure to work, we need an underlying persistent medium which can be the placeholder for messages so that publishers can publish to this common place from which the subscribers can pick up a message and consume them. In the current approach we use MSMQ as the underlying transport layer for the pub sub infrastructure. MSMQ is chosen as a viable option since it supports reliability, transaction, and order of messages.

Choice of Framework

We use the popular open source project Rhino Service Bus (RSB) which uses MSMQ infrastructure for transport of messages. We need to enable MSMQ by turning on the windows features which will be used behind the scenes by RSB. RSB has built in support for MSQM message handling and supports different kinds of Inversion of Control (IoC) containers, transport mechanisms, reliability, transaction, etc. We will be using Unity IoC container to configure our queues using the information in the configuration files.

We start by examining turning on MSMQ and then installing the RSB framework.

MSMQ installation

The steps shown here are the instructions to set up MSMQ on a Windows 7 image.

• Start->Control Panel->Programs and Features
• Click on "Turn Windows features on or off" on the left navigation
• Click on the Microsoft Message Queue (MSMQ) Server Core option under the Microsoft Message Queue (MSMQ) Server. This is the minimum feature which needs to be enabled for us to use MSMQ on the machine. This would require a restart of the machine for the changes to take effect.

Figure 2: MSMQ Windows Feature turned on

Rhino Service Bus

Rhino Service Bus can be obtained from https://github.com/hibernating-rhinos/rhino-esb and built with the build script, you would need to have GIT installed to get the source.

The reader can read up on the documentation of Rhino Service Bus, it has not been duplicated here. For the purposes of the current demonstration, we remember these simple concepts.

1. Rhino service bus lets publishers and subscribers communicate with each other using the syntax shown in the code examples.
2. A publisher can either "Publish" a message or "Notify" a message. If a publisher does not have any subscribers, a "Publish" operation results in an exception whereas a "Notify" operation does not raise an exception.
3. In the current implementation we use the Publish method to send a message, and use the Notify method to raise an event. When an event is raised, it's available on the bus to be consumed, if no one is interested, it is acceptable since there is no expected outcome when we raise an event.

Instead of using the GIT build we will use a simpler approach to use the pre built binaries in our solution. We will use a NuGet package to use the binaries.

The diagram below shows the inner workings of the MSMQ infrastructure used by RSB.

Figure 3: Message flow showing the publisher process and the subscriber processes

We can learn the following from the above diagram:

There are multiple processes (Publisher, Subscriber1, and Subscriber2) within any enterprise application. They communicate with each other via messages.

The application uses a Pub-Sub EAI pattern for message transfer. The two parties needed for any message transfer are a publisher and a subscriber. For a given publisher there can be multiple subscribers.

The subscriber initiates a communication request by letting the publisher know that it is interested in a message of Type T. This step is illustrated in the figure where Subscriber 1 process lets the publisher know that it is interested in a message of type Message 1 (Step 1a,b). Similarly Subscriber 2 process lets the publisher know that it is interested in a message of type Message 2. This is handled internally by the Rhino Service Bus; we add the different message types and the publisher/subscriber queue information in the configuration file. We go through a host and bus initialization steps which will correctly set up the message flow in the application. The sequence can be summarized as below:

• Initialize the host by creating a new instance of RSB DefaultHost.
• Initialize the bus. During this step, the publisher control queue creates a name value pair collection of subscriber-message type entries and stores it in the subscriptions queue. Thus if there are multiple parties interested in consuming a message of type T, the publisher makes multiple copies of message T and sends it to the different destinations. There is an AddSubscription message copy for each of these subscribers for a message of type T in the publisher's subscriptions queue.
• The diagram shows the message flow between two parties which assume the role of publisher and a subscriber. It is to be noted that their roles can be interchanged, or a subscriber process can function as a publisher in another scenario. The same rules of communication apply in those scenarios as well.

Proposed Solution

In order to illustrate the above communication scenario, we create a sample application the following projects.

Table 1: Projects used in the current solution

This solution will be built using Visual Studio 2008 and the .NET 4.0 framework.

List of queues: In order to simulate the processes indicated in Figure 3, we use the following queues in the application. These need to be created before using the application.

• PublisherUri – private queue used by the publisher process to which the messages are published.
• Subscriber1Uri – private queue used by Subscriber1 process which consumes a message of type T which was published at PublisherUri
• Subscriber2Uri – Subscriber1 process consumes a message of type T and raises an event of type E. Subscriber2 consumes this event at its private queue Subscriber2Uri
• SubscriberRepublishingUri – this is the queue used by Subscriber1 process to raise an event.

A queue can be created by navigating to Start->Control Panel->Administrative Tools->Computer Management->Expand Services and Applications->Expand Message Queuing->Right click Private Queues->New-> Private Queue-> QueueName->Check Transactional

Let us look through each of these projects. We will start with the class library which contains the implementation details for publishing a message.

CoolApplication.ESB

We need references of Rhino.ServiceBus and Rhino.ServiceBus.Unity which contains the implementation details of RSB infrastructure support. In order to add these references, we need NuGet plugin installed for Visual Studio. Once NuGet is installed the below steps will allow add the two references.

• Right click on the project name, and select Manage NuGet references.

Figure 4: Managing NuGet packages for a project.

• This will open the list of available NuGet packages. The text box on the right top corner lets us narrow down to the NuGet packages of interest. If you type Rhino.ServiceBus in the top text box, the list is narrow, install Rhino.ServiceBus and Rhino.ServiceBus.Unity from the list. This action will get any other dll dependencies.

Figure 5: Selecting Rhino.ServiceBus and Rhino.ServiceBus.Unity NuGet packages

Now that the references are added, let us pay attention to the wrapper and the publish method.

There are two main entry points for the helper class Publisher Publish and Notify. Both of them take a base class for all our messages in the application as an argument. They use a private method which does the heavy lifting of initializing the host, bus, and either publishing a message or raising an event. This is invoked on a new thread asynchronously.

Listing 1: Helper class which provides implementation details to publish a message and raise an event.

The above code snippet is pretty straight forward and uses the out of box Rhino Service Bus features. We would like to draw reader's attention to point to the step 2 in the above code snippet. The code looks for a file called Another.config file. This is useful for the cases where a subscriber wants to consume a message and republish it on a different private queue. The information for this queue is captured in the secondary config file for any subscriber process which also wants to act as a publisher.

The helper class and publish method is used in the Publishing host process. Let us look at the CoolApplication.PublishingHost application in detail.

The publishing host program has a simple implementation. The main program creates a new message, and uses the Publisher helper class to publish the message. All the underlying plumbing is encapsulated in the helper class and the publish method.

CoolApplication.PublishingHost

The publishing host code is shown as below.

Listing 2: Publishing Host process which creates a simple message and publishes it using the helper class.

The configuration information for the publisher is very simple too. For the publishing host, we need to provide a private msmq queue address to which messages can be published. The below xml shows the app.config contents for the publishing host.

Listing 3: Application configuration file for the Publishing host process

Next come the two subscriber processes. Let us look at the Subscriber1 process in detail. This process uses a BootStrapper which extends UnityBootStrapper. The entry point in the main program has the boiler plate code to create a host for RSB and initializing the bootstrapper. This will keep the subscriber process running and waiting for incoming messages on the bus. This is shown below.

Listing 4: Subscriber 1 process which creates a default host and initializes the boot strapper.

The bootstrapper code is very straightforward as shown below.

The process also contains a consumer class for the message "CreateOrder". This class implements the RSB interface "ConsumerOf<T>". The generic definition for this interface and the consumer code is shown below.

Listing 5: Subscriber 1 with implementation for the message consumer of type CreateOrder

Let us look at the app.config file for the Subscriber1 process.

Listing 6: Subscriber 1 app.config

There is a secondary config file called Another.config file which is used for the case where the message consumer republishes a message or raises an event. This file as shown below is exactly like the config file shown for the publishing host process, just that it helps us to configure a separate private queue for republishing a message. Since this is a custom config file, it will need to have the "Copy to output directory" property set to "Copy if newer".

Listing 7: Subscriber 1 Another.config.

The second subscriber process follows a similar strategy by using a bootstrapper, a message consumer, and a main program which creates a host and initializes the bootstrapper.

Listing 8: Subscriber 2 process and the consumer class which consumes the OrderCreated event from Subscriber 1 consumer.

Running the application

After all the projects have been set up as shown above it is time to run this application. The below diagram illustrates the flow of data between all the participants.

The applications need to be started in the order of Subscriber first->Publisher next. So using our sample set up, they need to be started in the order of Subscriber 2, Subscriber1, and then publisher. After this is done, you will see the message being consumed in subscriber1, republished and being consumed at subscriber 2.

The output is as shown below.

Figure 7: Output showing the message flowing from the Publisher->Subscriber1->Subscriber2

Summary

We have seen a sample distributed application set up between three processes. We covered the concepts on they need to be decoupled and how to use MSMQ as the underlying transport layer. We looked at creating a helper class which uses Rhino Service Bus behind the scenes used in publishing messages on the bus. We were able to have three processes communicate with each other in a decoupled manner using a message based architecture.

About Deepak Raghavan

Deepak has been architecting and developing enterprise applications for the past 14 years and currently works as an Architect in Minneapolis. His interests lie in the upcoming Microsoft technologies, SOA, SharePoint 2010, Agile and XP to name a few. He is on the executive board for the MN SharePoint... This author has published 3 articles on DotNetSlackers. View other articles or the complete profile here.