Mumbai University > Information Technology> Sem 8 > Software Testing Quality Assurance

Marks: 5M

Year: Dec 2016

Importance of Data Warehouse testing

• Organizations with already well defined IT practices are at an innovative stage to create the next level of technology transformation, by constructing their own data warehouse to store and monitor real-time data.

• They have realized the need to test this data to ensure data completeness and data integrity. They have also realized the fact that comprehensive testing of data at every point throughout the ETL process is important and inevitable, as more of this data is being collected and used for strategic decision-making that affects their business forecasts.

• But, certain current strategies being followed are time-consuming, resource-intensive, and inefficient. Thus, a well-planned, well defined and effective ETL testing scope guarantees smooth conversion of the project to the final production phase. Now, let us see some of the issues that are common with ETL and Data Warehouse testing.

ETL Testing

• An ETL tool extracts the data from all these heterogeneous data sources, transforms the data (like applying calculations, joining fields, keys, removing incorrect data fields, etc.), and loads it into a Data Warehouse.

• ETL stands for Extract-Transform-Load and is a typical process of loading data from a source system to the actual data warehouse and other data integration projects.

• It is important to know that independent verification and validation of data is gaining huge market potential.

Some of the important ETL Testing Challenges are:

• Loss of data might be there during the ETL process

• Incorrect, incomplete or duplicate data.

• DW system contains historical data, so the data volume is too large and extremely complex to perform ETL testing in the target system.

• ETL testers are normally not provided with access to see job schedules in the ETL tool. They hardly have access to BI Reporting tools to see the final layout of reports and data inside the reports.

• Tough to generate and build test cases, as data volume is too high and complex.

• ETL testers normally don’t have an idea of end-user report requirements and business flow of the information.

• ETL testing involves various complex SQL concepts for data validation in the target system.

• Sometimes the testers are not provided with the source-to-target mapping information.

• Unstable testing environment delay the development and testing of a process.