(An alliance-partner of Bender RBT Inc., NY, USA)
Software Testing: Defect prevention, defect detection and debugging
We have been applying the RBT process since the early 1970's. This has included just about every application type and technology including embedded to super-computer based applications and everything in between – e.g. financial, military, scientific, manufacturing, operating systems, communications, medical, etc. Over the years we have naturally refined it and improved the automated support. The following are brief discussions of the typical results achieved via this process from recent projects in the last year or so.
Manufacturing Support System
• System to manage design and production issues
• First iteration had >50% defect rate on initial test passes
• Second iteration switched to RBT
– Over 250 specification defects identified and resolved prior to development
– < 10% initial defect rate
– All testing completed in a single test phase
– No functional defects escaped into User Acceptance Test
– All tests automated immediately following first test phase
– 20% reduction in rework
• Same results in iteration 3 except initial defect rate < 5%
Hardware/Software Vendor Configurator
This is a system that ensures the compatibility of any hardware and/or software being ordered with the other items being ordered and with what is already installed.
• 60,000 business rules
• 25% change each month
• New releases multiple times per week
• No room for error
• Prior to RBT – numerous severity one and two defects per release
• Business rules modeled in RBT
• Updates to Configurator reflected in models
• RBT generates revised tests descriptions
• DTT add-on generates executable scripts
• Defect rate drops to near zero
• No severity ones or twos in last 3 years
• 12 people maintain 20,000 executable test scripts
Note: DTT (Direct to Test) is an add-on to the BenderRBT Test Case Design Tool developed by one of our partners. It is a framework tool that generates executable test scripts from the RBT output. It can generate those scripts to run on any of the major playback tools.
Credit Card System
• Credit card processing system for a bank
• Major inventory of customizations, little documentation, limited SME (subject matter experts) availability
• 60 models and about 900 test cases
• No customization functional defects detected by the user
• Over 3000 bugs in the base technology detected – i.e. the existing functionality that had to be traversed to get to the new functionality.
Bank Credit System
As stated earlier, most projects find that about 56% of the defects have their root cause in poor requirements. On this project the integration test effort found that only 2% of the defects could be traced back to requirements errors. This resulted in a huge savings in the time and costs to deliver the system. Once in production, no defects were encountered.