AI Ecosystem CourseLearn with Q
AI Ecosystem · In Development

Data: From Business Records To Model Inputs

See how business records become information an AI system can use, and why data quality matters.

About This Course

Introduction

An AI system receives specific inputs from a business workflow. It receives selected rows, documents, images, events, labels, vectors, and instructions. People and software choose how to collect, organize, transform, check, and share them. An apparent model problem may begin with one of those earlier choices.

We will start with records and databases, then move through pipelines, data quality, labels, synthetic data, retrieval, memory, rights, and governance. Your project is a data architecture for a business use case. Explain what belongs in training, what should be retrieved for a request, what the application retains as state, and why.

Use AI Foundations for data versus instructions and How Machine Learning Learns for training and evaluation splits. MLOps And The AI Development Lifecycle can help with versioning and pipelines. Lessons 1–6 cover storage and movement, 7–12 cover quality and learning, and the rest cover application use and control. Keep an orders-and-returns assistant in mind as a running example.

Planned Lessons

  1. A Record Represents Something, Under A ConventionUnderstand what a business record describes and how its values were collected.
  2. Tables, Keys, And RelationshipsSee how tables and shared identifiers connect information about customers, products or events.
  3. Queries, Joins, And Missing ValuesFind out how searching and combining tables can change which records appear in a result.
  4. Operational Databases, Warehouses, Lakes, And LakehousesCompare places to store data based on how the information will be used.
  5. Ingestion, ETL, ELT, And StreamingFollow records as they are collected, cleaned and moved for AI use.
  6. Pipeline Layers And LineageTrace where a piece of information came from and how it has been changed.
  7. Data Quality Is Fitness For A Particular UseCheck whether information is accurate and complete enough for a particular AI task.
  8. Sampling, Representation, And Selection BiasSpot people or cases missing from the examples used to teach a model.
  9. Labels, Annotation, And Ground TruthExamine how people decide the right answer attached to a training example.
  10. Pretraining, Post-Training, And Evaluation DataCompare the examples used to teach a model, improve its behaviour and test it.
  11. Synthetic Data: Useful Examples And Inherited ErrorsSee how AI-created examples can help training while carrying mistakes into new data.
  12. Proprietary Data And The Conditions For An AdvantageFind out when data that only one business holds can give it an advantage.
  13. Retrieval Data Is Available Evidence At Request TimeSee how an AI app looks up relevant information while answering a question.
  14. Embeddings, Vector Search, And Exact MatchingCompare finding similar meanings with finding an exact name, phrase or record.
  15. Memory And Application StateDistinguish information an app keeps between visits from the text sent to a model.
  16. Rights, Privacy, Access, And DeletionFollow who may use information and what must happen when it is deleted.
  17. Data Governance And Documentation That Can Be UsedKeep clear records of what data means, who owns it and how it may be used.