Linking data management to business intelligence

The success of BI depends upon accurate and well prepared data. Here are some tips to avoiding poor BI data influencing analytics results.

A constant issue IT faces is linking the work of data preparation with development work in analytics and business intelligence (BI).

Data quality is less of an issue when developing transactional applications because transaction programs abort when data is missing or erroneous. In these scenarios, the data has to be fixed.

It’s a little different in analytics and business intelligence work, as these apps are likely to keep running even if the data is wrong since the data edits within the software are likely to apply less data scrutiny than what is found in transactional programs.

Nevertheless, this doesn’t change the fact that accurate BI and analytics data is just as crucial as accurate data in day-to-day transaction processing.

When business intelligence and analytics programs process poor data, the likelihood of poor decisions based on this data increases. This compromises the ability for technologies like BI to make a positive impact on corporate decision making.

SEE: Microsoft Power Platform: What you need to know about it (free PDF) (TechRepublic)

How to avoid the influence of poor data in BI

To avoid analytics results being influenced by poor data in BI, it is important to enact an IT strategy that closely links data management with BI efforts. Here’s how this can be done:

1. Identify the degree of data accuracy needed for each BI application

In some cases, like analyzing weather reports over the last 100 years to determine long-term weather trends, it might be sufficient to operate at a data accuracy rate of only 70%. This is possible because only a general trend analysis is being done. But, if a weather report of far greater precision is needed, as is the case for understanding what the weather will be like for the following day’s drone mission, then a data accuracy of 95% or higher would be necessary.

That said, it can be difficult to determine how accurate data must be for each business use case. This is a decision that the BI applications group, the end users and the database groups should make upfront—before BI app development work is undertaken.

2. Align BI analysts and developers with data analysts in the database group

Data that is clean and accurate will primarily depend on the work done in the database group. It is the database group that stewards corporate data and moves data into new data repositories that BI operates on.

If the database group and the BI applications group operate in two mutually exclusive functional silos, it will be difficult for IT to link sound data management practices with the development of BI applications.

3. Prepare the data

With the help of the database group, BI developers can use tools like ETL (extract, transform, load) software to clean and format data correctly as it moves from various sources into the target data repository BI will use.

SEE: Best data science tools and software 2022 (TechRepublic)

Data preparation is a multistep process. It can involve identifying data that is broken, duplicated, in the wrong format, contextually irrelevant, etc.

The BI group and the database group should work closely together to identify all data and data forms that are unacceptable for each BI application, devising ways to either reform the data or exclude it.

4. Expect drift for BI and analytics applications

Over time, the data used for BI and analytics—and the business use cases themselves—get old. At least annually, IT should review the BI and analytics application portfolio with business users and with the database group to

1. See if business use cases have drifted away from original purposes, which will call for BI and analytics programs and data to be revised; and
2. See if the data being used by BI and analytics applications is still relevant or if it needs to be refreshed or revised.