AI Ecosystem CourseLearn with Q
Data: From Business Records To Model Inputs · Lesson 7

How To Check Data Quality For An AI Task

Judge information against the decision it will support. A quality result must name its rule, reference, population, time and consequences. A strong-looking aggregate cannot settle untested risks.

Study And Save Progress

Watch The Lesson

8 min 60 sec

Read the video transcript

Our fictional returns assistant receives a dataset described as 98% passing its checks. A customer then asks about an order, and the service connects that order to the wrong person. How can a reassuring quality figure coexist with a serious failure? We will investigate 1 delivery of order records, decide what its checks actually establish,

and choose what the service can safely do next. Data quality means fitness for a particular use. An order-status answer needs the right order, an appropriately current status, and permission to show it to that customer. A historical analysis may need an accurate picture of what was recorded at an earlier time. The same data can support 1 question and fail another.

Before choosing a quality measure, state the task, the people affected, and the consequence of an error. The UK Government Data Quality Framework, published in December 2020, sets out dimensions for examining fitness for purpose. Canada’s guidance, published in January 2024, also explains that different uses can give different dimensions different importance.

These are practical frameworks for their own public-sector contexts. They do not supply a universal score or a threshold that certifies our company’s assistant. Start with the reported figure. In this example, the source contains 1,500 distinct order records for the stated period. Only 1000 reach the receiving system. A required-currency check finds 20

received records with a missing currency. The other 980 pass that particular check. 20 divided by 1000 is a 2% failure rate among received records. It says nothing about whether all source orders arrived. 500 source records never reached this delivery. A source-to-received comparison exposes that gap: 1000 out of 1,500 arrived, about 2/3.

Then 20 of the received records failed the currency check. Keep those groups separate. The 980 passing records are records with a supplied currency under this rule. We still have to investigate their meanings, relationships and suitability for the task. Calling them clean would hide those unanswered questions. The U.S.

Census Bureau's May 19, 2022 results provide a real coverage check. Its Post-Enumeration Survey independently sampled households and matched them to the 2020 census. The national result had no statistically significant net coverage error, while 6 states had estimated undercounts and 8 had overcounts. The survey excluded group quarters and

Remote Alaska and carried sampling uncertainty. A reassuring aggregate can coexist with uneven coverage. For our service, inspect coverage at the level the intended decision needs, within what the evidence can support. Take order O701 from the passing group. Its customer identifier is C7, an identifier that exists in the customer source.

The relevant transaction reference for this invented case associates the order with C8. The customer link conforms to the identifier rule but disagrees with the appropriate reference. Validity concerns compliance with stated rules. Accuracy concerns agreement with an appropriate reference or the event being represented. A date such as the 31st of February

fails a date-validity check; a valid calendar date can still refer to the wrong transaction. A database constraint can reject some invalid values or missing required fields. PostgreSQL 18 documentation explains checks, not-null constraints and foreign keys as specific rules. Those mechanisms do not establish that every permitted customer identifier was assigned to the correct transaction.

In our case, hold the affected customer lookup, establish the correct relationship with authorized source evidence, and test the access path before releasing the answer. Changing how the customer’s name is spelled cannot repair this link. Now follow that order’s status through time. A preserved snapshot says shipped at 9 a.m. The source changes it to canceled at 10.

At 10:05, the assistant still uses the 9 o’clock snapshot. That snapshot may accurately describe what was recorded at 9. It is too old for this current-status question. For an investigation of what was known at 9, preserving it can be valuable. A historical prediction test must also avoid supplying later corrections as though

they were available at the prediction time. Useful freshness depends on the question and the required time boundary. The dimensions help locate different failures. Completeness asks whether the information required for the purpose is present. Consistency asks whether representations agree under a stated rule, including their meaning and time.

Uniqueness asks whether unwanted duplicates exist at the defined grain. Relevance asks whether the information bears on the task. A missing optional note might be acceptable for a status answer, while a missing currency prevents a trustworthy total across amounts. Neither a full record nor a fast update can compensate for another customer’s order entering the answer. Cleaning therefore needs decisions tied to these consequences.

2 copies of the same order under a verified source identifier may be unwanted duplication. 2 separate orders placed by a customer with the same name are different events. Standardizing their names and then merging by name can destroy a real distinction. An unusually large order also needs investigation; deleting it merely because it is

rare could remove a genuine event. Keep the original evidence and the transformation history under appropriate access and retention rules. Replacing a missing amount with an estimate changes the information used by an analysis. Record the assumption and test how it affects the intended result. For the current assistant,

prioritize the affected identity and status paths, with a defined response when information cannot be verified. For a historical demand model, examine coverage across the relevant periods and what was available at each prediction time. A pretraining collection raises further questions about duplication, languages, source quality, evaluation contamination and rights. Passing a currency rule answers only

a small part of those different needs. The practical response is a quality plan with an owner, a check, a consequence, and evidence for resuming the affected use. Recover and reconcile missing source records. Investigate currency failures before calculating a combined total. Verify the customer relationship and refresh the current status before answering this order question.

Choose acceptance rules with the responsible users and specialists, reflecting the consequences and operating requirements. No percentage in this lesson is offered as a threshold for another service. Here is the checkpoint. Why can an apparently clean dataset still be unsuitable for its intended use? Think about our passing currency check

and the customer waiting for an answer. Its selection, timing, rights, labels, coverage or business meaning may not match the task. A format check can pass while a customer relationship is wrong, a current status is stale, or part of the source population is absent. Decide which evidence the intended use requires before treating a quality result as permission to proceed.

Try a 2nd check. Out of 1,500 source records, 1000 arrive and 20 fail a currency check. What does the 2% failure rate describe, and what important gap does it leave out? It describes 20 failures among the 1000 received records. It leaves out the 500 source records that never arrived.

Report source coverage and received-record failures separately, then investigate both. The records that passed the currency check still need the other checks required by their use. For practice, choose checks for a new status lookup and a historical snapshot task. Trace a missing delivery, a valid but incorrect relationship, and 2 records that

look similar but may represent different orders. State what to hold, what evidence could resolve it, and who should decide whether the affected use can resume. The worksheet has a separate answer key. Our opening quality figure was too narrow for the service decision. A useful quality judgment explains the task, the tested property, the population covered and the consequences still unresolved.

Next we will examine how the collection process itself can leave relevant groups and situations out.

Main Point

Judge information against the decision it will support. A quality result must name its rule, reference, population, time and consequences. A strong-looking aggregate cannot settle untested risks.

Key Points

1. One delivery, one service problem

Continue the fictional order-and-returns service. Its source identifies 1,500 distinct order records for one specified period; the receiving system obtains 1,000. Each source identifier occurs once, and the source inventory is assumed complete for this exercise. Among received records, 20 lack a required currency and 980 pass that rule.

The currency failure rate is 20/1,000 = 2% of received records. Arrival coverage is 1,000/1,500 = 66⅔% of source records, with 500 absent. These are different questions with different denominators. The 980 currency-rule passes do not establish that customer relationships, dates or permissions are correct. No aggregate “safe-to-use” score follows.

If the source inventory itself were incomplete, this reconciliation could only describe that inventory. State the population and evidence used, including any uncertainty about its completeness.

2. Validity and accuracy can part company

Order O701 is in the passing group. It refers to customer C7, whose identifier exists. The appropriate transaction reference in this hypothetical case identifies C8. Referential validity can pass while the transaction relationship is inaccurate.

PostgreSQL 18 documents CHECK, NOT NULL, uniqueness and foreign-key constraints. Such controls enforce specified database rules; a permitted reference can still be assigned incorrectly by the source process. A CHECK expression evaluating to unknown can also pass, so requiredness needs a suitable NOT NULL rule rather than an assumed consequence of any check. These are PostgreSQL-specific implementation details, not a complete quality or authorization system. [PG-CONSTRAINT]

A malformed calendar date can fail validity. A valid date belonging to another transaction illustrates an accuracy problem. Choose an appropriate reference, investigate conflicts and preserve uncertainty when the reference cannot be established.

For O701, the affected customer lookup stays held while an authorized investigation establishes the relationship and tests the access path. Normalizing spelling cannot substitute for verifying identity. The lesson supplies no executed fix or observed service result.

3. Time changes the question

The preserved 09:00 snapshot says shipped. The source changes the status to canceled at 10:00. The assistant uses the old snapshot at 10:05. That snapshot can be accurate as a record of the earlier state and unsuitable as a current-status answer.

A historical question such as “what was recorded at 09:00?” may require that exact snapshot. A prediction test at 09:00 must exclude information first available at 10:00, including later corrections, unless its claim explicitly concerns retrospective revised data. Preserving history and updating the current view are separate responsibilities.

This establishes no blanket permission for historical model use. Still investigate coverage, rights, relevance, label definitions and the intended evaluation claim.

4. Use dimensions to locate consequences

Accuracy concerns agreement with a suitable reference; validity concerns specified rules; completeness concerns required information; consistency concerns agreement under defined conventions; uniqueness concerns unwanted duplication at the stated grain; timeliness concerns the required time relationship; relevance concerns the intended question.

The UK Government Data Quality Framework (3 December 2020) describes fit-for-purpose principles and six core dimensions. Canada's *Guidance on Data Quality* (10 January 2024) uses nine dimensions and explains that their importance varies with context. Their differing vocabularies support a practical point: this lesson's dimensions are useful diagnostic questions, not a universal exhaustive taxonomy or numerical score. [UK-FRAME; CA-GUIDE]

An optional note may have little consequence for a status query. Missing currency can defeat aggregation across amounts. A wrong customer relationship can create a disclosure risk even when most records pass other rules. Assess the affected operation rather than averaging away the consequence. Operational access controls and applicable rights remain necessary; a quality check alone grants no authority.

5. Cleaning is a decision with a history

For this service, compare source identity and event meaning before removing duplicates. A repeated copy of O701 differs from another legitimate order placed by a person with the same name. Standardization can improve consistency, but merging on standardized names can collapse distinct people or events.

A rare order may be genuine. Investigate its source before deleting it as an outlier. An imputed amount is an estimate under a rule; retain its status and evaluate sensitivity for the intended use. Missing values can affect a denominator, as the earlier query lesson showed.

The UK framework's guidance recommends action planning, root-cause investigation and communicating quality limitations. These practices motivate a specific local plan; they provide no universal error allowance for this assistant. [UK-ACTION]

A useful plan states:

  1. Intended use and affected people or decision.
  2. Property checked, reference, population and time boundary.
  3. Consequence of a failure and response for the affected operation.
  4. Owner and permitted investigation/correction route.
  5. Evidence needed before resuming that use, including regression checks and communicated limitations.

For the fictional delivery: reconcile the missing 500, investigate currency failures before relevant totals, and independently verify O701's relationship and current status before its answer. Acceptance conditions need agreement from responsible users and specialists. Any actual freshness limit or error tolerance requires a task-specific justification; this package invents none.

6. Practice and transfer

Use the blank worksheet to distinguish current lookup from historical snapshot, reconcile source coverage, and challenge a proposed cleanup. The separate key explains acceptable reasoning. The next lesson investigates how the collection process can omit relevant people, periods and situations even when received data passes local checks.

7. A dated operating example

The U.S. Census Bureau's May 19, 2022 results provide a real coverage check. Its Post-Enumeration Survey independently sampled households and matched them to the 2020 census. The national result had no statistically significant net coverage error, while six states had estimated undercounts and eight had overcounts. The survey excluded group quarters and Remote Alaska and carried sampling uncertainty. A reassuring aggregate can coexist with uneven coverage. For our service, inspect coverage at the level the intended decision needs, within what the evidence can support.

8. Bringing the lesson together

The opening 98% figure measured one rule among received records. You can now name the task, inspect source coverage and correctness, and decide which use must wait for evidence. A quality plan connects each defect to its consequence, owner and release check.

Learning Checks

Attempt each answer before expanding the reasoning.

M13-L07-Q01

Why can an apparently clean dataset still be unsuitable for its intended use? Think about our passing currency check and the customer waiting for an answer.

Show reasoning

Its selection, timing, rights, labels, coverage or business meaning may not match the task. A format check can pass while a customer relationship is wrong, a current status is stale, or part of the source population is absent. Decide which evidence the intended use requires before treating a quality result as permission to proceed.

M13-L07-Q02

Out of fifteen hundred source records, a thousand arrive and twenty fail a currency check. What does the two-percent failure rate describe, and what important gap does it leave out?

Show reasoning

It describes twenty failures among the thousand received records. It leaves out the five hundred source records that never arrived. Report source coverage and received-record failures separately, then investigate both. The records that passed the currency check still need the other checks required by their use.

Sources

Learner Materials

Download Lesson 7 Learner Materials

Use the supplied fictional cases and templates for practice.

Practice

Write your answer, compare it with the model answer and decide what to revisit. These written checkpoints are self-assessed.

Here is the checkpoint. Why can an apparently clean dataset still be unsuitable for its intended use? Think about our passing currency check and the customer waiting for an answer.

Show The Model Answer

Its selection, timing, rights, labels, coverage or business meaning may not match the task. A format check can pass while a customer relationship is wrong, a current status is stale, or part of the source population is absent. Decide which evidence the intended use requires before treating a quality result as permission to proceed.

Compare your reasoning with the model answer. These written checks are self-assessed.

Try a second check. Out of fifteen hundred source records, a thousand arrive and twenty fail a currency check. What does the two-percent failure rate describe, and what important gap does it leave out?

Show The Model Answer

It describes twenty failures among the thousand received records. It leaves out the five hundred source records that never arrived. Report source coverage and received-record failures separately, then investigate both. The records that passed the currency check still need the other checks required by their use.

Compare your reasoning with the model answer. These written checks are self-assessed.

Open Practice And Save Your Work