AI Ecosystem CourseBy Queensly Austin
Module 0 · Lesson 2

A Model Is One Component Of A Product

Evaluate the complete workflow, including documents, permissions and delivery.

Study And Practice This Lesson

Guest progress is saved in this browser. Optional email sign-in enables progress across devices once connected.

Watch the lesson

12 min 11 sec · English captions are available from the video controls and start off. To save your place and complete practice, use the learning app.

Read the video transcript

An employee is preparing for a work trip. She opens her company's travel assistant and asks how much she can spend on a hotel. A clear answer appears on the screen, with a link to the policy. What had to work for that answer to be useful? The system needed to understand her question. It also needed the right policy,

permission to use it, and a way to get the answer back to her. If any of those steps failed, a fluent sentence might still leave her with the wrong guidance. In Lesson One, we saw how a product can combine generation, prediction, decisions, and supporting steps. This lesson looks more closely at the difference between a model and the product around it.

We'll follow four steps. First, name the parts. Second, trace a task through them. Third, locate different kinds of failure. Finally, judge whether the whole product works, including what it costs to provide. Our recurring question is: what has to work for the employee to complete her task successfully?

Start with the model. A model is a computational structure that maps inputs to outputs. It receives information and processes it to return a result. Its architecture is its design: how its parts are arranged to carry out that processing. Its learned values are quantities adjusted through learning from data. Models can also involve other specified rules.

We don't need to examine those internal details yet. For this lesson, the key point is that the model performs a transformation from the information it receives to the result it produces. For our travel assistant, the inputs could include the employee's question and relevant policy text. The output could be a draft answer explaining the hotel allowance.

Now consider the application. This is the software product the employee uses. It surrounds the model with the features needed to make that output useful. A capable model doesn't automatically supply all of those features. Let's separate three more terms you'll hear throughout the course. The interface is how a person interacts with the system. Here, it's the screen where the employee types her question,

reads the answer, and opens the policy link. Buttons, messages, and the way information is presented are parts of the interface. The workflow is the sequence of activities that takes the task from beginning to completion. Receiving a question, checking access, finding a policy, producing an answer, and delivering it are steps in a workflow.

Infrastructure supplies the resources and operations that keep the service functioning. That includes computing, storage, networking, electricity, and the work of keeping the service running. The employee may never see those resources, but the application depends on them. We now have four useful names. The model processes inputs into outputs.

The interface lets the employee interact. The workflow connects the activities. Infrastructure keeps the service available. These names describe different responsibilities within the product. Let's follow the hotel question through the assistant. This is one possible workflow; products can arrange their steps differently. First, the product needs to know who is asking.

Signing in helps establish the employee's identity. Identity then matters for authorization: deciding what that person is allowed to access or do. An employee being recognized by the system doesn't mean she should receive every document the company holds. The product needs to apply the relevant permissions when selecting information and delivering results. Next, it needs the current travel policy.

It may retrieve a document, which means finding and bringing back relevant information. It needs a way to choose the version that applies now. A document being available doesn't establish that it is current. The question and appropriate policy information can then reach the model. That information supplied for the

response is part of its context. The model generates an answer using what it receives. The application still has work to do. It presents the answer to the employee and may include a citation: a reference showing where the information came from. A useful citation lets her inspect the source. A link by itself doesn't guarantee that the source

is current or that the answer represents it accurately. Other responsibilities support this sequence. Logging records relevant events so the team can investigate what happened. Cost controls help manage the resources a task consumes. Error recovery provides a way forward when a step fails, such as a document being unavailable. A support team can help with

problems the automated workflow can't resolve. Those responsibilities require design and ongoing operation. They don't disappear when the model gets better at generating language. We've named the parts and traced the task. The employee sees a question and an answer. Underneath, the product coordinates identity, permissions, information, model processing, and delivery.

Now we'll use that sequence to locate failures. Suppose the assistant retrieves a travel policy from 2022. The company changed its rules in 2025. The model accurately explains the old policy. The answer can match the supplied document and still be wrong for today's employee. We should investigate how the source was managed, retrieved, and selected for freshness.

Retraining means applying a learning procedure to adjust a model. Doing that may not solve a workflow that keeps supplying the old policy. We first need evidence about which step failed. Now change the situation. The current policy reaches the model, but the answer copies a number incorrectly. This time, the age of

the document doesn't explain the error. We would examine generation, how the supplied context was used, and whether output checking caught the mistake. These are possible places to investigate. Seeing a wrong number alone doesn't tell us the exact cause. Consider a third case.

The current answer is accurate, but it takes two minutes to appear. The employee gives up and contacts a colleague instead. The text might pass an accuracy test, but the task hasn't succeeded for that employee. Waiting time affected whether the product was useful. We need to examine the whole path to understand where the delay arose.

Finally, suppose a restricted document reaches an employee who isn't authorized to see it. The wording could be perfectly accurate. There is still an access-control failure. Access control is the set of checks that limits who can receive information or take an action. Here, the consequences extend beyond whether the answer is factually correct.

These cases give us different questions to investigate. Was the source current? Was the information represented accurately? Did the result arrive in time? Did it reach someone allowed to receive it? We shouldn't assume every problem needs the same model change. Naming the parts helps us choose what to inspect and then test

whether a fix addresses the actual failure. We've reached the fourth step: judging the whole product. A model benchmark measures model behavior under a particular test setup. For example, a test might supply policy passages and questions, then score the answers. That can tell us something useful about performance under those conditions.

The employee uses a larger system. Her real task involves documents that change, permissions that differ between people, waiting time, and an interface she has to understand. It also involves incentives: the practical reasons people act as they do. Someone trying to finish a booking quickly may abandon a slow assistant

even if its eventual answer is good. Product evaluation examines whether that complete system works for the intended task and user. We could test whether the assistant selects the current policy, excludes restricted information, answers accurately, displays useful evidence, and delivers the result to the correct employee within a useful time.

A model benchmark and a product evaluation answer different questions. Both can help. We need to know what was tested before applying a score to a real situation. A strong model result is evidence about one part under stated conditions. Successful use depends on the other parts and their connections too.

The same distinction matters when we look at the business providing the assistant. A model provider may sell access through an application programming interface, usually called an API. An API is a defined way for software systems to make requests and exchange results. The application company can send information to a model service and receive an output through that connection.

The model provider may charge for usage. The application company may charge its customers in a different way: by seats, completed workflows, or business outcomes. A seat usually means access for a user. A completed workflow means a task has reached an agreed completion point. An outcome refers to a result customers value.

What counts depends on the service and its agreement with customers. For our travel assistant, useful value might include helping employees find applicable guidance and finish their work with less effort. The application company needs revenue that supports the entire service. Its costs include model usage and other operating costs. It still needs the surrounding software,

infrastructure, document management, maintenance, and support. Strong model performance helps, but it doesn't establish that customers receive enough value or that the company's revenue covers its costs. We're keeping this at the level of the product. Later lessons will examine more of the businesses and resources underneath it. Let's test the distinction.

An assistant gives a faithful answer from an outdated policy. Which part of the task would you investigate first? Start with how the product manages, retrieves, and selects its sources. The model used the document it received. You need to check why an obsolete policy reached it before choosing a fix. Now try this checkpoint.

A model produces the right text, but the application sends it to the wrong recipient. Is the model's text-quality score enough to declare success? No. Successful completion includes the intended destination and the boundary around the action. The answer has to go to the person meant to receive it, under the appropriate authorization.

We would evaluate the product's authorization, interface, and execution path: the steps that actually carry out delivery. Correct text doesn't establish that those steps worked. Return to the employee asking about her hotel allowance. We can now describe what has to work. The model processes the information it receives into an output. The application coordinates the features needed to use that output.

Its interface connects with the employee, its workflow carries the task through its steps, and its infrastructure keeps the service functioning. When something goes wrong, identify the failure before choosing a fix. Check the source, the answer, the timing, the permissions, and the destination as appropriate. Evaluate the complete task alongside the model's performance.

Then consider whether the service creates value and earns enough revenue to support all of its costs. Choose an AI application you know. Describe one task a person uses it for. Name the model's input and output, then identify two responsibilities the surrounding product must handle. Explain one way that task could fail even if the generated text were accurate.

In Lesson Three, we'll look at how computers represent information and perform operations on it. That gives us a foundation for understanding the physical and software resources these products depend on.

1. The application adds the capabilities a model cannot supply alone

An AI product combines a model with the software and operating processes needed to complete a task. The model processes inputs and produces outputs using learned parameters or specified rules. The application provides the interface, connects the components and determines what happens to the result.

Suppose the app answers questions about your company's travel policy. Generating language is only part of the job. The product needs to identify you, find the current policy, check which documents you can access, and display an answer you can use. It may also need citations, logs, cost controls, error recovery, and someone to contact when it fails. The application needs software and operating controls for each of these responsibilities.

2. Diagnose the failing component before choosing a fix

The interface is what you interact with. The workflow is the sequence that takes your task from start to finish. Infrastructure supplies the computing, storage, networks, electricity, and operations that keep the service running. Each term points to a different part of the product.

Consider an assistant that accurately quotes a policy written in 2022. The company changed the rules in 2025. The model may have used its document faithfully, while the employee still receives the wrong answer for today. We would need to investigate the source documents, how they were retrieved, and how the current version was selected. Retraining the model might leave the actual problem untouched.

3. Evaluate task completion alongside model quality

Change the situation slightly. This time, the current policy reaches the model, but the answer copies a number incorrectly. We would look at generation, how the model used its context, and the checks on its output. If the answer is right but takes two minutes, the employee may give up before it arrives. If a restricted policy reaches the wrong employee, we have an access-control failure. The text can be good while the product fails the task.

Product testing must cover the complete task. A model benchmark tells us how the model behaved under a particular setup. A customer brings real documents, permissions, delays, interfaces, and working habits. Record the scope of each evaluation so its results can be interpreted correctly.

The same distinction matters commercially. A model provider might charge an application for API usage. The application might charge its customers per seat, workflow, or outcome. It has to pay for the model and everything else needed to run the service. Good model performance helps, but the business still needs to create value for customers and earn enough revenue to sustain the complete product.

Apply the idea

Test documents, permissions, response time, and delivery as part of the product.

Check your understanding

a model produces the right text, but the application sends it to the wrong recipient. Is the model's text-quality score enough to declare success?

Show the worked answer

no. Successful completion includes the intended destination and action boundary. The product's authorization, interface, and execution path must be evaluated.

Open Practice And Save Progress