CDO-1 Certificate Program: Foundations for Chief Data Officers. L6: Data Quality. Richard Y. Wang

Size: px

Start display at page:

Download "CDO-1 Certificate Program: Foundations for Chief Data Officers. L6: Data Quality. Richard Y. Wang"

Laurel Beryl Reed
5 years ago
Views:

1 CDO-1 Certificate Program: Foundations for Chief Data Officers L6: Data Quality Richard Y. Wang Executive Director, jointly with MIT CDOIQ Program Professor, UALR EIT College Director, MIT CDO and Information Quality Program Sept 26-29, 2016 (c)

2 Data Quality Foundation Without a solid foundation of high-quality data, dirty data can chip away at an organization s ability to function effectively. September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 2

3 We don t always notice data quality problems September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 3

4 Until September 26-29, 2016 (c) Contributing Faculty, 4

5 It s too late! Why is the water so low?! September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 5

6 What is Data Quality? Conventional View: Data Quality = Accuracy New Research Finding Data Quality Goes Beyond Accuracy September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 6

7 Why Is Data Quality Important? Data of high quality is a valuable asset Data of high quality can increase customer satisfaction Data of high quality can improve revenues and profits High-quality data is a strategic necessity September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 7

8 A Brief History of DQ Research at MIT September 26-29, 2016 (c) Contributing Faculty, 8

Early MIT Research (1988-1996) Define and establish data quality to be A multi-dimensional concept beyond accuracy Both objective and subjective Measure DQ with software tools such as Integrity

9 Early MIT Research ( ) Define and establish data quality to be A multi-dimensional concept beyond accuracy Both objective and subjective Measure DQ with software tools such as Integrity Analyzer and Information Quality Assessment Analyze DQ with models, methods & principles Modeling Information Manufacturing Systems to deliver highquality information products Improve September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 9

Journals * Books * - Information Quality & Knowledge (1999) - Data Quality (2000) -Journey to Data Quality (2006) Articles * - 1990 Polygen Data Quality Model (VLDB + ICIS) - 1996 Beyond Accuracy -

10 Journals * Books * - Information Quality & Knowledge (1999) - Data Quality (2000) -Journey to Data Quality (2006) Articles * Polygen Data Quality Model (VLDB + ICIS) Beyond Accuracy Managing Information as a Product, etc Evolution of Data & Information Quality ACM Journal on Data and Information Quality (JDIQ) Conferences and Certification Programs * International Conference on Information Quality (ICIQ) MIT-IQ program for Executives IQ-1: Principles and Foundations IQ Industry Symposium QIBRAS Conference, etc First CDO Symposium. Research Projects * Total Data Quality Management Program (TDQM) MIT Information Quality (MITIQ) Program Education MS IQ and IQ PhD Degree Programs * Not complete list IQ 10

11 Data Quality Practice Lessons Learned September 26-29, 2016 (c) Contributing Faculty, 11

12 Data Quality Issues People no longer ask What is data quality? They now say I know I have problems, but I don t know Real-time cleansing and matching E-business is real-time, so to should be the processes Meta data management Standards, definitions, and application meta data sharing Application-specific data and context DQ dashboards Top quantitative data measurements per function Rolled up into weighted metrics for entire organization Can be converted to $$$ showing cost savings September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 12

13 DQ Technologies Cleansing/Profiling/Auditing ETL (Extraction, Transform, Load) Real Time DQ check Business Rule Extraction Data Integration Name Matching Addressing Matching Entity Resolution Customer Recognition Systems for CRM Could Computing, Big Data September 26-29, 2016 (c) Contributing Faculty, 13

14 What We have Learned On the Corporate Agenda and intellectual Map DQ job titles un-heard of in early 1990 s, now given Academic and Commercial conference, workshop, journals Beyond Accuracy DQ a multi-dimensional concept Accessibility, believability, timeliness TQM may hurt if blindly applied 6-Sigma for DQ? Oh, no, I had it with TQM Systems, Processes, and Management are Important It is a Journey DQ in context Solution leads to new problem September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 14

15 The Road to Information of Quality Adopt a customer perspective Adopt and adapt classical TQM principles Manage information as a product It is a journey September 26-29, 2016 (c) Contributing Faculty, icdo@ualr 15

16 Journey to Data Quality (MIT Press, 2016) Discussion of book chapters (Book in the registration bag) September 26-29, 2016 (c) Contributing Faculty, 16

17 Journey to Data Quality Data Quality Definition Data Quality Measurement Managing Information as Product Data/Information Production Map Data Quality Policy September 26-29, 2016 (c) Contributing Faculty, 17

18 Questions & Answers (Q&A) September 26-29, 2016 (c) Contributing Faculty, 18

Smart cities: A human-centered approach Engineering and Construction Conference June 20 22, 2018

Smart cities: A human-centered approach 2018 Engineering and Construction Conference June 20 22, 2018 Agenda Topic Smart City Overview Content Drivers, Framework, Evolution Client Stories Success Factors