Implementing Custom Business Workflows using Rule Engine and Dynamic Form Rendering

8 October 2018

8 October 2018
Technology

Our approach to implementing custom business workflows for highest levels of dynamism and scale, by building business logic in the UI layer and building a rule management system that is capable of creating new workflows dynamically on the fly.

 

Introduction

With the ever-evolving business context that fast-paced organizations operate in, business and operating models are re-invented very often. This underscores the need for the ability to change underlying workflows and rules in applications with ease and agility.

This is a core tenet of our engineering fundamentals at RIVIGO. In order to understand this well, let’s consider the example of our asset management system. At any given point, there can be several thousands, even millions of unique assets operating on both our relay-based logistics network and the marketplace platform. There are more than 150 distinct asset types and more than 300K points that are collected across asset categories like the engine, tyre tube, rim etc. As context evolves, constant modifications in asset categories, data points, workflows and rules are rightfully expected.

Every truck consists of more than 500 other assets that are required for maintenance/working of the truck. Each of these assets has a separate life cycle. Let’s consider the tyre as a unique asset category within the truck. There are around 10 tyres in every truck. Multiplying by fleet size, the number of unique assets becomes huge. In the event of puncture/rupture/ageing/re-trading, replacement of the tyre is required. This means that a single tyre may be used in X different trucks in its lifespan. To store a tyre’s information in the system, several variables need to be considered. Category, tube status, pattern, mileage (km), rim size, brand, manufacturer, tyre pressure (PSI) are some of them. The system built here is required to digitise all the assets and track any changes happening to them (maintenance/relocation etc.) while considering the nature of different asset types, their lifecycle and workflows.

To sum it up, the system is required to handle the following.

  • Different workflows for different assets as handling in application makes it complex
  • Business logic in UI layer to handle workflows for different asset categories
  • No development and deployment for incremental asset type addition
  • Requirement of a rule engine to create business logics dynamically on the fly
  • Nested complexity and undefined dimensions
  • Ability to handle frequent changes and deployments
  • Faster turnaround time for production

 

Approach

Keeping in mind the complexity and to build for scale, we decided to use a rule engine which defines business rules not in the code but outside it, i.e., in the database. This was required so that even a non-technical person could define or change rules directly with the interface without any deployment effort.

To put it simply, rules are a set of condition-based actions. They direct the system in the following way, ‘when some condition occurs, then perform some actions’. Generic logics are written on the application side and specific business rules are defined in the rule engine to control the following.

  • Defining state machine: State transitions based on business rule
  • Dynamic interaction with UI: Forms are produced dynamically using custom JSON schema, which had to be smart enough to derive any possible asset through it
  • View and Access control
  • Validations
  • Business process automation

As shown in the diagram above, microservice creates and uses KIE session to connect to the business rule interface engine. All the facts (objects) are inserted in the session with optional agenda groups based on categorization. After inserting these facts to the working memory, when rules are fired from KIE session, the pattern matcher uses facts from the working memory and rules from the rule management service (custom microservice to manage and handle life cycle of rules). Pattern matcher uses Rete algorithm to match facts against rules and working memory is modified (insert, update etc.) based on the logic.  Any microservice can use rule-modified working memory to handle any workflow.

Microservice has to define one-time sample template in application resource. To onboard any microservice for the rule engine, the rule engine library has to be imported. It takes the microservice specific template and rules from the rule management service. We can also define multiple placeholders in the template to generate rules dynamically, where placeholder values are filled with externally stored parameters. This could include agenda group, when logic, then logic etc.

Below is a brief structure of this template.

Line 1:  All templates start with heading “template header”

Lines 2-8: Header is followed by a list of columns in the order they appear in the data

Line 9: Empty line signifying the end of the column definitions

Lines 10-11: Contains package statement, imports and global definitions

Line 13: The “template” keyword signals the start of a rule template. There can be more than one template in a template file. The template should have a unique name.

Lines 14: The “rule” keyword signals the start of a rule followed by unique name for that rule

Line 15: Agenda group is grouping of rules (ex: validation rules)

Line 16: Salience in priority of rule if multiple rules needs to be fired

Lines 17-18: When conditions

Lines 19-20: Actions to be performed if condition matches

Line 21: “end” signifies the end of a particular rule

Line 22: “end template” signifies the end of the template.

Rule data is compiled with the given template. It creates rules for each row of data by replacing @{param_name} of template with value of param_name of that rule. This subsequently creates a DRL file automatically which is passed to the KIE system of drools. This creates a KIE session and stores these rules in the working memory of the application.

Given below are the firing rules.

  • We need to pass the object and set of agendas for which rules have to be fired
  • Actions will be performed on the passed object for rules which satisfy the ‘when’ condition

Sample code to load rule in the memory at runtime is given below.

Once we get stateful KIE session from the KIE container, the sample code given below is used to fire rules for object of agenda group “live_field_rules”.

Finally, to dispose the session, the following is used.

Whenever any rule changes in the rule management service, the rules (in memory) are refreshed with new rules without any deployment.

 

Rule management service

Rule management is a lightweight microservice which handles the lifecycle of rules. The user defines rule on the interface based on access. Rules are then translated to a custom criteria template, which has all the fields, operators and child components. This can be used to store nested conditions.

All rules are stored in mongodb with specified group type and agenda group (which is customizable based on microservices) in extensible format so that rules can be rebuilt and changed easily. Executable conditions are derived from the criteria template and stored in mysql. These are used by drools to load into the working memory. The code block given below shows a sample criteria template object.

 

Front End Architecture and Technologies used

Architecture

To achieve the complex functionality described above, we needed a way to render pages that are controlled by configurations supplied from the backend.

We even needed a way to apply all the rules designed by the backend rule engine. Therefore, we decided to render pages dynamically with each field acting as a hot field. Any change that is happening to that field is captured and checked with the rules engine for any custom logic, if it exists.

All the logic is then translated to actions which are performed by a web page.

Let’s again consider the example of a tyre as an asset. It has various fields such as brand, manufacturer, model, ply, width, height etc. When a user selects MRF as brand, then the models corresponding to MRF are populated. When a particular tyre model is selected, then the fields such as ply, width, height etc. are auto-derived from the rules.

To take care of such dynamism, we needed to design the JSON structure of a single element in a way that it fulfilled the criteria mentioned below.

  • Grouping of multiple elements to form a section inside a form
  • Ability to manage conditions when changes to a component lead to change in some other components (example, change in tyre brand may autofill rim size)
  • Configurable visual themes
  • Validations
  • Custom error messages

The final JSON structure came out to be like this.

Now, an array of such objects can be used to create forms for any use case. In this case, we generate various assets forms, using which the entire inventory and assets get digitized.

This is how the application looks like now with every part being controlled via JSON schema. Even the navigation bar is dynamic to accommodate different asset hierarchies for different asset types.

Technologies Used (UI front)

 

Conclusion

Keeping the rule management system outside of the code and building it for the highest levels of dynamism and scale has helped product managers create rules on the interface directly. This has not only enabled them to evolve the product at pace without tech intervention but has also saved time and repeated development for engineers. Similar approaches have been taken across services and products at RIVIGO to help enable agility and scale.

 

Aman Juneja, Lead Frontend Developer at RIVIGO, has co-authored this article.