What FLOPS Tell You About AI Speed

An umbrella retailer wants to speed up its nightly forecast. One accelerator advertises a very large FLOPS figure. Another sounds smaller on paper. Can the retailer rank them from those two numbers and predict how many useful forecasts it will finish? Only if the numbers describe comparable work, and even then the complete service still has to be measured. Lesson six: FLOPS and the meaning of a peak. By the end, you should be able to explain the unit, check the conditions behind a peak claim, use a sustained rate for a bounded time estimate, and distinguish performance per watt from energy per useful task.

We'll follow three steps. First, define the numerical rate and its qualifiers. Second, work through a hypothetical ten-trillion-operation example. Third, connect the measured rate to quality, power and the retailer's complete task. The guiding question is: which rate was quoted, under what conditions, and what actually gets delivered? A FLOP is a counted floating-point operation, a unit we met when we counted matrix work. FLOPS means floating-point operations per second. It is a rate: how many operations of a specified kind are counted in a second.

A trillion such operations per second is a teraflop per second in ordinary performance language. The unit is precise only when we also know what operations and counting convention are included. In Lesson three, one fused multiply-add instruction could count as two operations under a common convention. A different convention could produce a different number for the same work. FLOPS does not mean correct answers per second. The retailer's forecast must still be useful, meet its quality threshold and arrive when needed. An AI application also contains operations that are not captured by a simple floating-point count: data handling, communication, control and waiting. A large FLOPS number can describe a powerful numerical capability without describing the whole job.

Peak throughput is a theoretical or specified maximum under stated conditions. It is a ceiling for some supported work, not a prediction that every application will run there. Start with numerical format. A claim for one precision may differ from a claim for another. Then ask whether the figure assumes dense or sparse work. Check how operations were counted. Check the hardware path and clock conditions used for the figure. A quoted rate from a supported specialized engine may not apply to every operation the retailer's model requests.

Imagine that one claim uses a smaller format and structured sparse operations while another uses dense work at a different format. Comparing their headline numbers as if they measure the same job would be unfair. First align the assumptions. Documentation can help explain the boundary of a claim, but the retailer must still profile its actual workload. A vendor's comparison remains a vendor claim unless it is evaluated independently under matched conditions. Dense operations treat the relevant values as broadly present. Sparse work has selected values or operations that can be skipped. Structured sparsity adds a required pattern: the missing work must occur in a form the hardware and software know how to exploit.

A device may advertise a high sparse peak because, under the required pattern, it can avoid selected calculations or count effective work according to a stated convention. If the retailer's model does not have that pattern, or the software does not use the supported path, the sparse figure is not automatically available. The lesson is not that sparse work is bad. It is that a peak number carries conditions. The retailer needs to identify the actual model structure and the program selected to run it. If those differ from the claim's assumptions, the large number is a poor basis for a forecast of service time. Sustained workload performance is the numerical rate measured while useful work runs under the intended conditions.

It can be far below a qualified peak. Data may have to move to the chip. Some operations depend on earlier results. A kernel may be inefficient for the model's shapes. Devices may communicate, and other stages may wait. Even a high allocation percentage is not proof of high achieved arithmetic throughput. A device can be reserved or appear busy while it spends time on memory traffic, coordination or operations outside the advertised fast path. We need the measured rate of the relevant work, not just evidence that capacity was assigned.

For the retailer, we would define the batch, number format, quality requirement and software version. We would measure the numerical stage and then the complete forecast process. Those boundaries matter: the chip's arithmetic rate, the model's execution time and the time to deliver a useful forecast are related but different measurements. Let's use a hypothetical task. It needs ten trillion counted operations. Suppose the relevant sustained rate on a device is five trillion counted operations per second. Divide the amount of work by the rate.

Ten trillion operations divided by five trillion operations per second gives two seconds. The operation units cancel, leaving seconds. That is a simplified compute-stage estimate. It assumes the rate is actually sustained for this work and that the counted operations describe the part we are estimating. It does not automatically include data preparation, retrieval, communication, queues or writing the result. Now suppose the same device has a peak claim of one hundred trillion operations per second. Ten divided by one hundred gives one tenth of a second on paper. But treating that point-one-second quotient as the retailer's forecast time would be unjustified

unless this exact workload can reach the peak and the other relevant delays are represented. The two-second estimate uses a relevant measured rate; even it keeps a clear compute-only boundary. The order of the calculation matters. Define the counted work. Choose a rate measured for comparable work. Divide with units. Then check what time remains outside that boundary. A larger denominator is useful only when the task can actually use it.

Performance per watt divides a specified performance rate by a specified power boundary. Is the denominator power drawn by the chip, a complete device, or the whole server? A figure from one boundary cannot be compared fairly with a figure from another without adjustment. Energy per task asks how much energy is used to complete a defined task. Power is a rate of energy use. If, purely as a hypothetical example, the device drew a steady five hundred watts during a two-second compute stage, that stage would use one thousand watt-seconds, or one thousand joules, at the device boundary. It would not count the CPU, memory outside that boundary, networking, waiting, or energy spent on unsuccessful tasks.

A chip can have attractive peak operations per watt and still leave a poorly mapped application with high energy per successful forecast. The retailer needs energy and cost per task that meets quality and service targets. That measurement brings the hardware claim back to the organization's useful outcome. Name four conditions to align before comparing two peak FLOPS figures. Align the numerical format, the dense or sparse assumption, the operation-count convention, and the relevant hardware and clock configuration. Then test the same useful workload at the required quality and measure the result. A larger unmatched claim does not establish a faster service.

One calculation check. At a sustained five trillion counted operations per second, how long is ten trillion counted operations in the simplified compute-only estimate? Two seconds. Ten trillion operations divided by five trillion operations per second leaves two seconds. It is a compute-stage estimate under the stated rate, not the retailer's complete elapsed time or the number of correct forecasts. FLOPS is a rate of specified floating-point work. Peak FLOPS requires assumptions about format, sparsity, counting, hardware path and clocks. A sustained workload rate can support a bounded time estimate, but the complete task also includes movement, software, communication and waits.

Performance per watt and energy per qualifying task describe different boundaries. For a purchase decision, the retailer should match claim conditions, profile its actual workload and compare useful outcomes at the required quality, service time, energy and cost. In Lesson seven, we'll ask a different physical question: can the complete working set fit in available memory? A fast arithmetic rate cannot rescue a workload that cannot be placed and run under its memory needs.