Big Data Requires Testing the Data Warehouse

Everywhere you look, you see the term “Big Data.”  Marketing, human resources, operations and, seemingly, every other function within business will rely on big data to make fact-based decisions in the future.  While the jury is still out on what the term means for each function, it is clear that rapidly dropping costs in computing power and memory are making it much easier to track and analyze all kinds of data to facilitate predictive modeling.  Even the NSA is trying to take advantage of big data to monitor electronic communications to predict and stop terrorist activities.  Behind the big data concept are powerful computer programs that try to make sense of the data.  One of the precursors to ensuring these programs work properly is to perform data warehouse testing.

Until only a few years ago, a testing team had to develop testing criteria and business cases manually, then take test data samples from production data. It was labor, time and capital intensive.  Today, there are automated testing programs that enable a non-technical person to define the business value of each test result so that automated programs can focus on tests that make the biggest difference to the functionality of the program.

In addition, the automated testing programs allow for the creation of synthetic data for testing the data warehouse.  Automated testing using synthetic data is particularly useful in highly regulated industries, such as banking and insurance.  Your team can set specific test cases that meet regulatory requirements and maintain the results of those tests to show to regulators upon request.

The latest automated testing schemes allow you to focus on the business rules that govern your data warehouse.  They allow non-technical people to use plain-English to define tests based on these business rules.   They can then run the tests using automation, allowing technical staff to focus on other tasks.

Big data is now easier to analyze with automated data warehouse testing.

November 21, 2015 | Category: Business | Tag: