http://blog.boyangyue.com/tags/data-quality/Recent content in Data Quality on Boyang YueHugoen-usA Good Year AheadMon, 07 Oct 2024 10:00:00 +0900http://blog.boyangyue.com/2024/10/asserting-observing-and-owning-data-quality/Mon, 07 Oct 2024 10:00:00 +0900http://blog.boyangyue.com/2024/10/asserting-observing-and-owning-data-quality/<p>A nightly pipeline can pass every check it runs and still hand a dashboard the wrong number. The schema matches, no required column is null, row counts fall inside their usual band, and the revenue total is quietly wrong anyway, because an upstream service changed what one field means while leaving its type and shape untouched. The checks were green. The data was not correct.</p> <p>Running checks is not the same as having data worth trusting, and the distance between the two is seldom one missing rule that a longer list would have caught. It is more often a matter of framing. A validation suite is a model of how data can go wrong, and a suite can be diligent, extensive, and green while modeling the wrong failures. Three different concerns travel under the single name data quality, and they are easiest to keep apart by what it takes to check each one.</p>