Business Rules and Engines

This topic expands on my previous post “Increasing STP Rates – the Fantastic Four“. Specifically, I’m going to expand on the role of the centralised Rules Engine, what it does and how to use it. ().

You’ll no doubt have heard about these before and have perhaps even worked with something like Drools or the Wolf Framework. I’m not going to deliberate the best technology stack here, the best choice for you in that regard will have a lot to do with your current stack, your ability to support that stack and your appetite for 3rd party solutions. Instead, I want to hone in on business rules, how you can classify them and how you can look to implement them.

Our experience has taught us that rules generally fall into a number of predefined classifications:

• Data handling (completeness, transformation and mapping)
• Enrichment (internal domain information, allocation, matching)

• Filtering (inclusion and exclusion)
• Sequencing (expected order, missing messages, duplication and replay/resubmission)

• Validation (successes and breaks)
• Routing (single/multiple destinations, re-routing, replay)

All of these classifications present their own specific challenges. Their correct application is critical to the success of enterprise messaging solutions (our speciality). Regardless of whether you are working on a green-field project or replacing an existing spaghetti middle/back office solution, you need to elicit and classify your business rules correctly.

Let’s run through the classifications and see the challenges that each presents.

Data handling & Enrichment
All messaging systems need to implement these rules. You need to ensure that the message you are processing is correct both syntactically and symantically. Does it conform to your XML schema? Are the required data items present for this business case? Can you consistently map the incoming message format to your domain language? Can you enrich your data to simplify downstream processing?

Ideally, these business rules need to be performed as early as possible – at your system boundary. If you allow malformed and incomplete messages onto your bus via your routing rules, your downstream consumers with feel the pain, your problems will grow expotencially and your costs will increase significantly. Advising your Operations Managers that they will need to throw bodies into the middle office to manually resolve data problems is not a viable career development strategy.

Filtering & Sequencing
Often, your system will be presented with message that are out of sequence, duplicated or that your system does not need to process. Perhaps worse is the scenario where you are not presented with messages that you expect to receive.

You need to understand the conversations that take place in your incoming messages. Do you just receive and process unique, isolated messages or do your messages convey business meaning through a chain and sequence (new booking, reversal, shaped adjustments etc).

Filtering rules will protect your system from data that it isn’t required to process. Sequencing rules will identify missing, out of sequence messages and intelligently manage them until the missing messages arrive and the sequence can be played correctly or until such time as they decide that this is not going to happen and alerts need to be raised.

Validation & Routing
Your rules need to be able to specify the action to be taken upon a successful or unsuccessful outcome. If this message has been handled and enriched; filtered and sequenced and all is well, what do you do with it then? Where does it need to go? Does it need to go to more than one place? What if it has broken one of these rules? What happens next?

The big challenge facing business rules and the rules engines they operate in is that, more often than not, several business rules relating to a single incoming message will be identified. In order to process the message successfully, these rules need to be applied in the correct priority sequence so that business meaning can be achieved. If the first rule fails, then the rules engine will move onto the next rule and will keep going until a match rule has been found. However, be watchful for distinct subsets in these rules that require a hard failure only halfway through the rule stack. Your rules engine must be able to handle this scenario.

Whichever way you decide to implement business rules, the above will be invaluable to achieving success in your project. Before I finish, here are just a few parting shots – make sure that:

• Your business rules are reusable
• Your rules engine can access external data
• You have adequate logging
• You have automated regression testing packs that can verify rule changes quickly
• You have sufficient granularity in your rules (client/product specific)
• You have adequate versioning and documentation of your business rule sets

There, I think that just about does it. Business rules are a fantastic direction to take your project in. Just be very careful about how you approach designing and implementing them. And remember, the only thing more expensive than a good rules business analyst is a bad one.

()