What Workplace Studies Show About AI

Imagine a customer-support manager reading that an AI assistant improved productivity by fifteen percent. Her team has experienced advisers, new recruits and customers who need accurate answers. She wants to set a new output target and decide where to invest in training. What does the study give her a reason to do? This is Lesson four, What Workplace Studies Show About AI. We will work from that reported result to a decision the manager can defend, including the evidence that would change it.

First she needs to know what improved. In Generative AI at Work, published in 2025 by Erik Brynjolfsson, Danielle Li and Lindsey Raymond, the outcome was customer issues resolved per hour. A resolved issue connects the worker’s effort to a useful customer result. The assistant provided response suggestions during conversations, and agents could edit or ignore them. The study followed five thousand one hundred and seventy-two agents at one firm. These details tell our manager which work and outcome sit behind the number.

The authors estimated an average productivity increase of fifteen percent. The distribution is important for the manager’s decision. Less experienced and lower-skilled workers gained more. The most experienced and highest-skilled workers had small gains in speed and small declines in conversation quality. If she gives everyone the same fifteen-percent target, she discards information that could help her support newer staff and protect service quality.

She needs to examine the workers and outcomes separately. The design suggests a possible explanation to investigate. Suggestions and documentation can make useful practices available during a difficult conversation. The authors interpret their results as consistent with useful practices spreading to workers with less experience. Our manager could ask whether new recruits struggle to find the right procedure and whether timely suggestions address that difficulty.

She should also ask why an experienced adviser might follow a suggestion that lowers answer quality. The study has now led to specific training and tool-design questions. Before investing, she asks how the researchers separated the assistant’s contribution from other changes. The tool arrived at different times for different agents. The authors used this staggered introduction to compare changes in outcomes around access to assistance.

This gives a stronger investigation than simply observing the firm’s performance after launch. Its interpretation still depends on the comparison, earlier patterns and the conditions the researchers examine. The manager needs those details to judge how much weight the estimate can carry. Consider a problem in her own team. Suppose the assistant arrives in the same month as a better knowledge base and additional training.

Resolved issues per hour then rise. The records show an improvement in the combined process. To learn which change contributed, she needs a comparison that can separate their effects. That matters for her budget: buying more software, improving sources and training staff are different actions. The explanation behind the improvement should guide the next investment. One option is a randomized intervention. Assigning access or conditions by chance can create groups with which to estimate

the intervention’s effect, when the design and execution are sound. She would still check participation, compliance with assigned conditions, missing outcomes and whether the tasks remained comparable. A natural experiment instead uses a circumstance that creates a comparison, with assumptions specific to that setting. An uncontrolled before-and-after record tracks change over time while leaving simultaneous changes entangled. Understanding the design tells her which competing explanations have been investigated.

Even random assignment leaves questions about whose work entered the study. In a February 2026 update, METR described difficulties in its follow-up developer experiment. Some developers and tasks were selected out because participants were reluctant to work without AI. Working with several AI agents concurrently also made time records difficult to interpret. The researchers judged the resulting productivity signal unreliable. For our manager, the lesson is practical: record who participates,

which cases enter the trial and how time and completed work are measured. Those choices can change the result she sees. She can now design a more informative support pilot. It would define accepted resolutions and quality criteria, compare similar work under a suitable design, and examine outcomes by experience. It would record system version, task mix, participation and training. Agents would have a clear way to reject a suggestion, and reviewers would examine customer outcomes as well as speed.

These are proposed choices for her investigation. The published support study supplies reasons to ask these questions; her local evidence would show how they play out in her team. The manager also needs a decision rule. If newer advisers resolve issues more effectively at the agreed quality standard, she might extend assistance with suitable training. If experienced staff’s quality falls, she might change how suggestions appear or revise their use before expansion.

Staff should help identify mistakes and work displaced into checking. Customers’ successful resolutions and treatment belong in the evaluation. An average rate is useful when it leads to these concrete comparisons and decisions. Try the first Learning Check. Can a result from one customer-support study be applied to every occupation? Explain what limits that inference, even if the study’s own comparison is well designed. Pause and state a specific reason. Tasks, workers, systems and conditions differ across occupations.

Even within the support study, workers with different experience had different outcomes. Examine whether the mechanism and conditions carry over to the proposed setting. A credible local comparison helps explain that setting; broader claims need evidence from the other workers and tasks involved. Now try the second Learning Check. What should accompany a workplace productivity number? Name the information you would keep with it so another person can judge what the number supports.

Pause and construct your evidence card. Keep the task, worker population, system and period. Add the study design and comparison, the outcome and acceptance standard, exclusions and limits of generalization. Record participation, task selection and relevant simultaneous changes. These details explain how to interpret the number and which decision it can inform. Return to the support manager. She began with a fifteen-percent headline and a plan for a uniform target.

She now has a more useful proposal: examine where suggestions help newer advisers, investigate quality for experienced staff, and choose a comparison that can inform training and tool design. The published study measures medium-run effects in one firm and leaves aggregate wage and employment effects outside its observed scope. The manager’s own decision concerns an identifiable service, workers and customers, so she can specify the local evidence and the observation that would change her plan. For Practice, audit the different fictional pilot in your worksheet.

Fill in the evidence card, identify missing information and write the conclusion the description supports. Then name a comparison or additional record that would improve the investigation. Make your attempt before using the separate answer key. You can now move from a productivity claim to its measured result, the comparison behind it and an action with a clear revision condition. The next lesson asks who receives the value when work changes: how skills, bottlenecks and team organization affect wages and opportunities,

and what helps people navigate the transition.