Institutional investors want more data, but not just any data – they want more useful data. In an article published on BNP Paribas, Thomas Durif, chief data officer for the bank’s Securities Services, explains the importance of qualifying data and the impact it can have on efficiency.
Qualifying data means to describe and enrich the data; to know its origin, definition, owner lifecycle, confidentiality classification, and level of quality, writes Durif. It is about pulling together information about the data – also known as metadata – so that it can be organised, found, and understood.
A good foundation
Durif lists several reasons why data qualification is important. One is to prevent data duplication, which can happen when data is “created at different times for varying purposes”. This highlights the necessity of aligning data dictionaries.
Another reason is automation – only when the data populating systems is structured and trustworthy, can the issue of “garbage in, garbage out” be overcome. “Leveraging generative AI’s full potential will require data to be far more accurate and transparent,” Durif points out. “Providers must also be able to qualify data within an organisation if they are to train models purely with authorised data and in a fully transparent way vis-à-vis their clients.”
Keep it simple, but not too simple
He identifies three “core pillars” for qualifying data: documenting the information, quality control, and avoiding pitfalls.
One of the most common pitfalls is overengineering. “In a world of growing data volumes and complexity, data qualification needs to strike the right balance between being simple to use and understand, while being sufficiently detailed to add value. The temptation is to create a data catalogue that is beautifully engineered but that may be too complicated for actual users,” Durif cautions.
He describes qualifying data as a “sisyphean task”. “Keeping business objectives front of mind is essential in managing boundaries in a way that brings value to users and avoids qualification becoming overly exhaustive.”
