Is Your Organization in Data Denial?

By Joe DeSiena - President, Consulting Services - Bardess Group Ltd.  

Common Problems with Corporate Data

Research has shown that the amount of data and information acquired by companies has close to tripled in the past four years, while an estimated 10 to 30 percent of it may be categorized as being of "poor quality" (i.e., inaccurate, inconsistent, poorly formatted, entered incorrectly, etc.). The common problems with corporate data are many, but typically fall into the following five major areas:  

• Data Definition - typically manifesting itself through inconsistent definitions within a company's corporate infrastructure.

• Initial Data Entry - caused by incorrect values entered by employees (or vendors) into the corporate database, typos and/or intentional errors, poor training and/or monitoring of data input, poor data input templates, poor (or nonexistent) edits/proofs of data values, etc.

• Decay - causing the data to become inaccurate over time (e.g., customer address, telephone, contact info, asset values, sales/purchase volumes, etc.).

• Data Movement - caused by poor extract, transform and load (ETL) processes that lead to the creation of data warehouses often comprised of more inaccurate information than the original legacy sources, or excluding data that is mistakenly identified as inaccurate, inability to mine data in the source structure, or poor transformation of data.

• Data Use - or the incorrect application of data to specific information objects, such as spreadsheets, queries, reports, portals, etc.

Each of these areas represents a potential problem to any business, both in their existence within the organization, as well as the ability of the organization to even recognize that the problem exists. In any case, these are classic symptoms of "data denial" - one of the most costly economic drains on the well-being of businesses today.

Data Quality Maturity Levels

There are five key status indicators that can be used to measure the existing levels of Data Quality maturity in an organization, each with its own set of distinct corporate - and human - attributes. However, it is at the mature level where you will want your organization to be positioned.   

• Embryonic - this level is the least beneficial place to be, as Data Quality does not even appear on the organization's radar screen; there is extensive finger-pointing with respect to data-associated blame, generally leading to cover-ups and CYAs; and there is no formal Data Quality organization in place. As far as the humans involved in the process are concerned - they are totally "clueless".

• Infancy - this level is not that much better, although the organization has begun to consider looking into Data Quality; various ad hoc groups may have been established to search for "answers"; and Data Quality has been positioned as a subset of corporate IT. This typically occurs as the human element begins to show an emerging interest.

• Adolescence - this level is one of mixed Data Quality accomplishments where most of the pain points have already been identified and the strategy team has shifted into a crisis-driven "full court press" managed by formal Data Quality teams that are populated and coordinated by both IT and the Business. However, this is also the point where alternating periods of panic and frenzy typically set in.

• Young Adult - by the time the organization reaches this level, there begins to be some semblance of an evolving Data Quality structure, where the entire organization is involved, one where both IT and the Business have begun to work as partners toward a common goal. Accordingly, the human attribute has also become much more "stabilizing".

• Mature - once the organization has attained this level, it has finally reached the point where it has implemented an effective Data Quality structure, characterized by collaborative efforts and Data Quality/Center of Excellence (DQCE), as well as the ability to measure and track customer value over time. As such, the organization has been able to attain a "controlled" environment, where all of the personnel involved - on both the supply and demand sides - are comfortable that the desired levels of Data Quality have been achieved.   

Part 3 Next - Moving Towards Data Quality

Deploying an Enterprise-wide QlikView Architecture 

By Philip Duplisey - Senior Manager, Consulting Services - Bardess Group Ltd.  

At Bardess, we have chosen QlikView as our preferred Business Intelligence solution.  Over the years in this business, we have finally found a software product that lives up to its promise of ease of use, rapid development, low cost of ownership and ultimately end user self service BI.

Having said that, QlikView is the new kid on the block, and most of our clients are large F500 companies that have complex environments and expect secure, scalable solutions. So the question becomes, is QlikView ready for the enterprise? It has been our experience that, yes, QlikView can go head to head with the mega-stack vendors like IBM Cognos, SAP Business Objects, and Oracle who have dominated the enterprise business intelligence landscape in the past.

A QlikView deployment can scale from a single desktop user to tens of thousands of users across a global enterprise if required. Refer to the diagram below from QlikTech's white paper - "Deploying QlikView'.

With the advent of 64 bit Windows servers, the key is clustering multiple QlikView servers and QlikView Publisher to build a deployment that can scale to support several thousand users, provide resiliency, failover, load balancing, and full QlikView analysis. Each deployment is unique, and architected to suit the unique environment and business requirements of the end users. Contact Bardess and we will help you design and build a QlikView environment to meet your business intelligence needs.

June 2011