Want to know what it takes to work in data and analytics? Every other Friday, our own Informationists will share their thoughts, experiences, and advice on what they do and what they did to get there. Expect to see a wide range of answers from individuals in the same lines of work; our Informationists come from all walks of life, which only shows that there’s more than one way to get on the right career track.
Our first post featured Dip Kharod, Big Data Architect. Continuing the series is Reed Bradford, Data Quality Architect. Reed earned his MBA in Computer & Information Sciences from Temple University and his BS in Information Management from Brigham Young University (BYU). He has worked in information management and analytics for 28 years across the financial services, life sciences, hospitality, manufacturing, and retail industries.
What does a Data Quality Architect do?
I help organizations get a handle on the quality of their data to help them achieve their business goals. If you think about, most businesses need to be data-driven in this age of advanced analytics, to some degree or another. So this is a huge opportunity to have a direct impact on the success of just about any business. I apply proven Data Quality Management techniques and tools to measure, control, monitor, and improve the quality of data needed for key business processes.
What educational and/or professional background does a Data Quality Architect need?
Working with data quality requires a unique blend of business, data management and technology education and experience. An understanding of business is important because data quality really depends on the business context and the requirements of the business. Without a focus on the business, Data Quality Management becomes more of an academic exercise that may turn out to be of little value to the business.
I was very fortunate to have majored in Information Management at BYU in the early 1980s. I was a member of the first graduating class for that major at BYU. It was a new major focused on combining a solid business education with data management and a small dab of computer science. I remember the Computer Science students used to make fun of us because we didn’t take some of the more technical courses, like how to build a CPU. However, in retrospect, the Information Management degree was much more valuable because it grounded me in business and data management fundamentals – something you don’t get with a pure Computer Science degree. It also paid off when companies came recruiting at BYU. At the time, those with Information Management degrees did much better than those with the Computer Science degrees because the hiring companies knew we had the business and data management fundamentals. Obviously, keeping up on the technical side is very important too, and I continue to take training courses to stay current.
What skills do you need to be a Data Quality Architect?
First, you need to understand the business and what is important to the business so you can help them focus on the data that is most important to them. I think you have to be able to talk the language of the business, down to the data entity and attribute level. That includes an understanding of the industry as well as the specific business processes you are dealing with (such as Marketing or Finance).
Second, you need solid data management skills. This begins with an understanding of Data Governance principles and how Data Governance relates to Data Quality Management. Solid data modeling skills are very important. The quality of data is highly dependent upon the quality of the underlying data models because those metadata definitions supply many of the data quality rules. The ability to query and analyze data in a database management or file systems is also essential. Finally, a mastery of how data quality is measured is particularly important, including understanding completeness, uniqueness, validity, accuracy, integrity, timeliness, etc.
Finally, you need a broad range of technical skills because Data Quality Management can span all types of technology platforms across an entire enterprise: front-end real-time applications for data entry, data integration platforms (e.g., Extract, Transform & Load (ETL), Enterprise Information Integration (EII), Enterprise Service Bus (ESB), Web Services), batch data processing applications, operational data stores (ODS), data warehouses (DWs), data marts, Big Data repositories and Business Intelligence/reporting/analytics platforms.
What tools and technologies does a Data Quality Architect use?
I always like to stress that Data Quality Management is all about the tools. Yes, people and processes are important too, but they should be built around leading-edge DQM tools. If you are not using among the best DQM tools, I think it is almost impossible to manage data quality properly, regardless of the people or the processes you try to throw at the problem.
You must master technologies and tools specific to Data Quality Management: Data Discovery & Profiling, Data Quality Rules Management, Developing Reusable Data Quality Validation, Cleansing and Standardization Components, and Data Quality Monitoring and Reporting, including Dashboards and Scorecards. I think the most successful Data Quality Architects know how to get the most out of these technologies and tools to help the organizations they are working for. Often, these technologies and tools are not used at all, are underutilized, or are misused, leading to very poor DQM results.
What are the common traits of the best Data Quality Architects?
Well, first of all, I think you have to be very detail oriented. This is not something a typical “big picture” kind of person is going to excel at. Next, I think you have to be a very good communicator. Regardless of your education, background or technical skills, if you cannot properly gather the data quality management requirements and then communicate the data quality management solution, you are not going to be successful. This cannot be stressed enough. Discussions about data quality often span an entire organization, from senior level executives, to business and technology managers, to more junior data stewards and technical programmers/analysts.
I think the best Data Quality Architects have the ability to perform root-cause analysis to develop the best DQM solutions. DQM solutions can involve complex business process or technical platform changes that get to the heart of the data quality problems. Finally, tenacity and patience are required. Improving data quality across an organization is not an easy thing to do and requires constant effort and diligence over sustained periods of time.
What are your favorite and least favorite parts of being a Data Quality Architect?
I really enjoy seeing the light bulb come on in people’s heads as they realize what can be accomplished with proper Data Quality Management. Often, they have not heard how to properly manage data quality with the right people, processes and tools and the possibilities of affecting very positive change in their organization is very exciting to them. It can have a tremendously positive effect on business results.
I guess the least favorite part is many people seem to think that Data Quality Management is someone else’s problem. In reality, it is everyone’s problem and you must convince people in key roles to become accountable for data quality across the enterprise.
What advice do you have for someone looking to become a Data Quality Engineer?
Having spent almost 30 years in Information Management now, I think that many organizations are just now starting to “get it” and the field is wide open now. The opportunities in Data Quality Management are almost limitless. Many of the other Information Management & Analytics disciplines depend on solid DQM: Master Data Management (MDM), Data Warehousing (DW), Business Intelligence (BI), and Big Data and Advanced Analytics.
I would suggest setting career goals to provide the foundation you need to be a successful Data Quality Architect and then to continue to get trained on DQM platforms and stay current. You should also look for opportunities to gain on-the-job training and experience with data-driven companies.
Have other questions or comments for Reed? Post them in the comments!