Testing Methodology For Business Rules-Based Systems
When it comes to building
There have been some new approaches to the software development life cycle (SDLC) in recent years. These new approaches have included new testing methodologies. The Agile and Extreme Programming (XP) communities have gone so far as to advocate that testing drive development. In these environments, prior to any development, tests are written as the software is being created. Software is only created in response to a failed test. If there is no failed test, there is no software to write.
The last few years have witnessed the advent of business-rule based systems. Rules-based systems require a very different approach in that the rules are the system. Rules-based systems (or at least portions of them) can either be generated by a rules repository or the behavior of the system can be driven by the rules in the repository at runtime. How can testing be done in such an environment?
This paper will attempt to answer that question. It includes discussion of some traditional testing mechanisms and how to apply them to a rules-based software development environment. Description of a real world project where this modified methodology was employed is also included.
Testing in Traditional Systems
In a traditional environment, testing usually takes place late in the process after the Strategy, Analysis, Design and Development phases are complete. When testing does take place late in the process, what is really being tested is an almost complete system that is built on a series of completed phases, none of which may be based on a sound foundation or successfully satisfy user requirements. This is not a recipe for success.
If no testing takes place until the system is built, how can users and developers be sure that the earlier phases have been completed successfully? Have major areas been overlooked in the Strategy phase? Did Analysis paralysis set in requiring endless additional modifications that may or may not be necessary for a first version of the system? Is the data model sound enough for design and development to take place?
The Analysis phase requires that developers gain an understanding of the business that the system intends to support. It is often very difficult to ascertain when analysis is complete. Often hundreds of pages of documents are created as part of this phase. At this point, users have no mechanism for validating the business rules collected by the analyst. They are at the mercy of the analyst, hoping that they were able to effectively communicate the business functions.
Analysts take their interview notes and other documents gathered and create the system specifications for the developers, who go behind closed doors to code the system. No testing or validation of the system specifications is done at this point. Users are not able to effectively evaluate the analysis documents in order to determine whether or not their requirements have been met. It is only after the developers have coded the system that users are given an opportunity to validate their requirements.
Figure 1 illustrates the user interaction with the system in a traditional development environment. There is no common point of communication between users and IT professionals.
Figure 1: User/IT Professional Interaction in a Traditional SDLC
When the “finished” system is presented to the users, they are faced with the daunting task of trying to reverse-engineer the original business rules from the working system. One can easily argue that this development methodology is flawed. There is a critical disconnect between the users and the IT professionals during much of the development process. The result is very often that large time and money resources are expended to produce a system that may or may not do what the users intended or wanted.
The next section discusses the differences in the SDLC of rules-based systems and how the approach to testing is handled in these types of system development efforts.
Testing in Rules-Based Systems
The process of building rules-based systems is quite different from that of traditional systems development. Rules-based systems normally include some type of repository populated by business rules. Once the rules have been validated, it is possible to generate a working system, which then is ready for user feedback. This iterative process can be quickly repeated many times as part of the testing process. In this way, users are kept involved in all aspects of the SDLC and much more immediate feedback can be provided to ensure that the system being built meets the user requirements.
This section describes the different phases of a Rule-Based System
Development Life Cycle (RBSDLC) and how testing is performed in each phase.
I. Analysis Validation
Analysis is one of the critical phases of the RBSDLC since the foundation of the system is based upon this analysis. If the analysis is flawed, then the system has a high probability of failure. Since users are one of the biggest stake holders of the system, the real success or failure of the system depends upon user acceptance. In all phases of RBSDLC, close contact is maintained with the users by formalizing the input given by users and minimizing the time between system versions that the users can test.
In the analysis phase, rules are collected using traditional techniques of interviewing. In addition, Use Cases offer a convenient and easily readable method of performing analysis. (For a more complete description of this process, see “Business Rules Analysis with Use Cases” on the Dulcian website at http://www.dulcian.com/papers/Business RulesAnalysiswithUseCases_revised.htm)
Instead of users explaining their business functions to the analyst who then creates the data model and analysis documents, in the RBSDLC using Use Case modeling, users can be taught the Use Case vocabulary to identify the relevant Actors, Use Cases, and Scenarios. Once users are comfortable with the working vocabulary, they can create actual Use Cases. To ensure a complete specification of the business rules, the system analyst can probe further with questions such as:
·
Is the Use Case complete? Are there any details
that need to be added?
·
Do you feel confident that the actor's goal is
going to be properly met?
·
Are there any procedural or requirement changes
that would simplify the process depicted in the Use Case?
·
Are there any additional goals of the actors
that are not addressed?
· Are there any additional actors that are not represented (directly or indirectly)?
As part of the analysis process, the system analyst must also
mine the business rules from the legacy system, if one exists. This process also serves to validate the
Using these techniques to supplement the traditional SDLC Analysis process is a better way of understanding the business environment while helping to ensure the quality and accuracy of the process of collecting the appropriate business rules and system requirements.
II. Design Time Validation
In a rules-based system, much of the user validation takes place in the design phase, which is quite contrary to a more traditional development process. The design team builds the system along with the users. Keeping users involved in the design phase can help avoid many bad design decisions early in the development cycle, saving both time and money.
The system design notation used in most traditional SDLCs is user-hostile. This means that users are not able to provide meaningful feedback during the design process. Using a rules-based approach, the business rules gathered are divided into structural and process rules. These rules are then used for the implementation process but their format can be read and evaluated by users as well as IT professionals. System designers provide guidance about which rules are classified as structural and which are process rules.
The design validation process includes visual inspection of the design by the users. Figures 1 and 2 illustrate the design of both the structural and process rules. The design notation used here is feature-rich and at a higher level of abstraction, making it easier for the user to understand and communicate effectively with the design team. The design team can use these diagrams to walk the users through the design by describing the different business functions represented by the diagram that are of interest to a particular group of users.
Figure 1: Structure Design
Figure 2: Process Design
Using Dulcian’s Business Rules Information Manager (BRIM®) the designs shown above can be quickly
and easily converted into implemented rules. This allows users to test the
system right away without waiting months for the development team to turn the
In addition to the huge advantages from the users’ perspective for building systems this way, there is also a tremendous cost savings. Systems build using an RBSDLC are much more flexible and can adapt more easily to changes in an organization’s business rules. This, in turn, increases the potential lifetime length of the system being built and reduces ongoing maintenance costs over time.
III. Implementation Validation
The BRSDLC Implementation phase differs significantly from a traditional SDLC because in a rules-based system, the rules are the system so that there is very little hand-written code required to implement the system. Once the business rules themselves are gathered, categorized into structural and process rules and modified somewhat into a format that is machine readable, the system can be generated automatically.
Traditional unit testing for such systems is not necessary since a significant part of the system is generated. For the parts of the system which must be hand-coded to handle special business functions, traditional testing techniques such as unit, integration and regression testing can be performed. In some cases, Extreme Programming (XP) techniques can also be employed for quality assurance in the hand-written code. The XP “test first” techniques are not relevant to the parts of the business rules-based systems that are generated since the time interval between design and implementation is close to zero.
IV. Performance Validation
Achieving adequate system performance is always a challenging task. In an RBSDLC it is possible to begin performance measures in earlier prototypes of the system as compared to traditional development where performance testing does not even begin until the system design is solidified in the fourth or fifth prototype.
The tools for performing load and stress testing are the same for both types of development life cycles. Once the system is generated, it can be handled like any other system and performance tested using traditional methods.
V. Other Validations
Although the process of using a rule-based methodology to analyze, design and implement systems has many benefits, it also has some overhead in the area of testing. The first important area which requires extra testing is the business rule engine itself, including the rules repository and code generators.
Although the rules engines may well tested, some of the nuances of a business-specific design can produce unpredictable results which lead to bugs in the generated code. It is very important to closely monitor the generation process at all times. While testing the implemented code, developers should not exclude possible engine faults from the scope of testing. If testing limited to validation of the business rules and their implementation, there is a possibility of intermittent bugs being present due to engine faults.
The code generators themselves are quite complex. These generators take quite a while to mature and produce quality code. A great deal of testing is done to make sure that the generators create the appropriate code which also needs to be validated.
The second area which should be included in testing business rules-based systems are any factors which may be environment-specific. Developers can extend the rules engine to support new business rules and these extensions should be tested like any other hand-written code.
Figure 2 illustrates that user involvement in the RBSDLC. In this case, the rule repository becomes the common point of communication between users and IT professionals.
Figure 2: User/IT Professional Interaction in a Rules-Based SDLC
Example of Using the RBSDLC to Build a System
Recently Dulcian worked on a recruiting system for the United States
Air Force Reserve (USAFR). The core of the system involved the automation of
thirty-seven different forms. These forms are completed in different
combinations by many types of users.
Analysis was simultaneously performed in three areas. First, a group of users created use cases for their business functions. Second, the legacy design of the system was examined and mined for business rules. Third, communication was established with the IT professionals who had worked on the previous version of the system. While the business rules were being collected from all three sources, the initial design of the system was created. This would have been impossible to do in a traditional development environment.
Analysis Validation
To validate the analysis phase, the data model was audited by other developers who considered the following questions:
· Do the structural and process models correctly handle the complex business transactions?
·
Does the model have the capability to store all of
the
· Does the data structure support the most complex questions?
Design-Time Validation
The system design was structured by partitioning the business rules into structural and process rules. Within two weeks of the initial analysis, a first version of the system was ready to show to users. After a system walkthrough, users were able to validate some of the business rules. Using the RBSDLC allowed us to compare and merge the business rules collected from the three different sources in the analysis phase, thus validating major parts of our initial design.
The result of validation process included modifications made to the data model and identification of any business rules that could not be captured within the model. There were deliberate decisions made earlier in the modeling process to explicitly exclude some business rules in the structural model because of the extra complexity they would have added to the system. We revisited all of those decisions and defended or modified the structural rules in the audit/validation process.
The key to the success of the RBSDLC is to be in constant contact with the users. Educating the users in understanding the design notation and keeping the version turnaround time to a minimum helps to validate the design significantly faster. The meaningful feedback from the users on a continuous basis ensures that the system meets their requirements. Design sessions often include hours of discussion with the users about how to implement different business functions. Making users an integral part of the design eliminates many of the possible communication errors about the business process present in a traditional SDLC. For this reason, the business rule environment is much more conducive to the ultimate success of the system.
Implementation Validation
Once the design was completed, acceptance testing of the first prototype began. It was much easier for the users to give meaningful feedback by using a working system than by trying to read through hundreds of pages of analysis documents. During the development of this particular system, we used a two-week design/development cycle.
To provide traceability of the implemented business rules in the finished system, use cases were linked to the implemented system. Each business rule is linked to an appropriate structure element. In this example, shown in Figure 3, the business rules surrounding the filling out of time sheets, including approval of time sheets, maximum number of hours that can be worked etc. are linked to the timesheet class through a rules repository application. This process helped to validate where each of the business rules was stored and how they interacted with other business rules. Going forward, if any business rule needs to be modified, an impact analysis can be conducted on the rule repository to determine the cost and what areas of the system are directly or indirectly affected by this rule modification.
Figure 3: Sample User Interface for Tracking Implemented Rules
Conclusions
This paper compared the traditional SDLC testing methodology with the modified RBSDLC in a business rule environment. It should be kept in mind that a traditional SDLC and a RBSDLC are very different from each other. If an organization is planning to a rules-based project, they should take adopt the new RBSDLC and no try to build this type of system using a traditional SDLC. The RBSDLC testing techniques should be used to help ensure a successful project.
It should always be kept in mind that the ultimate goal of testing is to validate and verify the existing business rules implemented in the system and to expose any missing requirements for the next version. As long as this philosophy is adhered to during the iterative development process, the chances of project success are significantly increased.
About the Author