10 Things You’ll Do On Every Messaging Project – Designing an Adapter Framework

This is the second in our series on life on an enteprise messaging project. This time we will be discussing that adapter framework you noticed appearing in the development team’s conversations. There’s no line for it on the project plan, and the client is *definitely* not going to want to pay for it.

DISCLAIMER Nothing in this article should be taken as a inducement or encouragement to engage in enterprise architecture unless supervised by a fully-qualified adult. The only thing more expensive than a good enterprise architect is a bad one.

What are adapters?

Adapters are the integration components that connect your messaging software to your core systems. You will have two basic types:
– ‘Outbound’ adapters: These poll your core systems for changes, and convert these changes into messages which they push to the messaging software. E.g. you might watch a SQL table and convert new records into a message
– ‘Inbound’ adapters: These poll a message queue for new messages, and convert these messages into API calls to a core system (assuming you’re lucky enough to have an API – the default one is called ‘stored procedure’)

What are adapter frameworks?

Adapter frameworks turn up as soon as you realise you are going to be creating quite a few adapters in your architecture. There are lots of things that ALL of your adapters will be doing, so it makes sense to solve these problems once, and then share these common solutions across all adapters. The rest of this article is about how long that list can get.

Logging and Error Handling

You can never have enough logging in a software system that doens’t have a GUI. You want to make it *really* easy for adapter developers to send frequent and verbose messages to the logging layer. You want log messages to include things like time, date and machine name automatically. If you allow multiple instances of the same adapter, you also need to correlate log messages to a particular instance. You might also consider rolling your log messages up to a single location, then you can get a unified view.

Error handling starts with logging – but you then should think about some general error handling strategies that you wish adapter developers to adopt. You should have a common set of negative unit tests that all adapters should pass:
– queueing system is down
– core system is down
– adapter host is out of disk space / memory
– poison message

You should also consider sanitising any messages that either support staff or end users will encounter. This could be addressed by a  as much as a code library.

Architecture

Unless you have a particular reason to allow cross-platform adapter development, you will want adapter code as either Java libraries or .NET assemblies, which in turn implies development languages, run-times and operating systems. Both are safe choices – any core system you encounter will probably have:
(a) A binary SDK for both platforms -or-
(b) A platform-netural API that both platforms support (e.g. SOAP, ODBC)

There are many standards that are relevant to the messaging world:
– AQMP for messaging
– JMX or SNMP to allow your adapters to report status
– WCF as an adapter accelerator if you are using .NET
– Common Logging
– Quartz for adapter poll schedules

As ever, it is more important that you have standards than the particular standard you select.

You should consider at what level different adapter instances share resources. Any combination of the following strategies are possible
– multi-threading inside a single adapter instance
– multiple adapter instances inside a single container (an example of a container would be Tomcat in the Java world)
– multiple containers inside a single run-time (either the .NET run-time or the Java run-time)
– multiple run-times on the same machine

These choices have an impact on manageability and thus your support department. A lot of support and monitoring software is targeted at the machine level, so you might think you are saving money on virtual machine instances by running multiple adapters on single nodes, but you are certainly going to be spending some money replicating or modifying systems like Hyperic.

Adapter Management

On larger projects, you will find that the challenge of configuring, deploying, versioning, controlling and monitoring adapters looms large. It makes sense to centralise as many of these functions as possible. Your adapters will need to be self-describing, and will ideally be discoverable to management systems, rather than needing to be explicitly registered.

There will be some configuration differences between an adapter polling a queue and an adapter pushing records into an SQL database, but these differences should be a matter of different fields being present, rather than requiring radically different administration procedures. Thanks to the  , configuration settings will always include both a description and examples of valid values, so that it is easy for support teams to pick up what is going on with a new piece of software.

Adapter Business Logic

While your development team are building your adapter framework, your business analysts will be busy analysing and defining the actual messages that will shortly be winging their way through your adapters. Most analysts will present these specifications in either Excel or Visio (if you are lucky), but the foward-thinking architect may wish to consider embedding an ETL design tool such as Volante into the adapter framework. That way, specifications can be created directly in the GUI, and can be exported back into documentation at any point when paperwork is unavoidable. You may not be able to persuade your analysts to touch these tools, but they will certainly be a godsend to the development team.

You will also find that some message types are taken in and out of adapters in multiple places, particularly if you are thinking ahead and using a standard such as FIX. So you should also consider sharing your ‘message library’ across adapters. That way, updates can be applied consistently, or mistakes can be fixed in just one place (depending on how jaded you’re getting by now.)

Conclusion

With all this work going into the adapter framework, you’d be forgiven for wondering if this isn’t where most of the actual effort goes, and you’d be right. You’re actually creating an application template that may get used 10 or a 100 times through your project. But if you get it right, you’ll be creating and deploying new adapter instances in matter of weeks.