Are We Making Good Decisions?

The tools have gotten better. Forecasting models are more sophisticated than they were a decade ago. Data pipelines are faster. Dashboards have more panels. Revenue management systems have moved from mainframe terminals to cloud-hosted platforms with more modern UIs. The investment has been real, and the technology has genuinely improved.

But I am wondering: have the decisions gotten better?

Not the tools. Not the systems. Not the adoption metrics or the uptime figures or the number of markets covered. The actual decisions—the sale price set on a Tuesday morning flight, the competitive response filed across overlapping markets, the outperforming flights discovered by an Analyst where inventory was tightened. Are those decisions better than they were ten years ago, in proportion to the investment made in the infrastructure supporting them?

I have spent enough time inside airline commercial operations to believe the honest answer is: we do not know, and that this is an expensive problem.

What Gets Measured

Airline commercial teams measure a great deal. Revenue against target. Load factor against prior year, peer markets, reference curves. Yield variance. RASM/RASK. System uptime. Forecast accuracy at the aggregate level.

What they almost never measure is decision quality at the individual level.

Nobody is tracking whether the protection level an analyst set on a high-demand market last Thursday was the right one—not whether the flight closed full, but whether the controls were correctly calibrated given what was knowable at the time. Nobody is systematically reviewing whether the competitive fare response filed on a Monday morning was warranted or whether it was reactionary and revenue dilutive. Nobody is going back to the market flagged to push fare, reviewing what action was taken, and asking whether the outcome validated the action.

This is not a criticism of analysts, it’s just a structural observation. The feedback loop does not exist in most commercial operations, not because people do not want it, but because building it requires connecting systems that were never designed to talk to each other, and nobody has prioritized that connection over the daily work of running the operation.

Why This Matters

Here is why the absence of that feedback loop is so costly: without it, the system cannot learn, and neither can the people inside it.

An analyst who makes two hundred decisions a week and receives no structured feedback on whether those decisions were good has only one source of calibration: intuition built over time from patterns that may or may not be reliable. That intuition is valuable. Experienced RM analysts carry genuine pattern recognition that no model has fully replicated, but intuition without feedback is also how confident errors persist. A miscalibrated mental model about how a specific market behaves, a systematic bias toward over-protecting in certain competitive situations, a tendency to dismiss signals that have historically been noise but have recently changed character—these are exactly the kinds of errors that structured feedback would surface and that the absence of it allows to compound.

The same problem applies at the system level. An RMS that takes actions that are never evaluated against outcomes has no mechanism for improvement. It does not know which actions produced the expected result, and it is uninformed about extraneous demand impacts, high fuel prices, or temporary commercial strategy shifts. It is generating outputs into a void.

The AI Question Hiding Inside This One

Every conversation about deploying AI in airline commercial operations eventually arrives at the same set of questions: which data sources, which agent architecture, which signals to surface. Those are the questions that need to be answered first.

A level deeper lies this question: if we cannot currently measure whether our decisions are good, how will we know whether the AI is making them better?

This is not a reason to avoid the investment. It is a reason to make a different investment alongside it. The infrastructure for closing the feedback loop—connecting actions to the outcome—is not glamorous work. It does not make for a compelling vendor demo and it can make people feel vulnerable or uncomfortable, but it is the foundation on which any honest claim about AI-driven improvement in commercial operations has to rest.

An AI system that surfaces better signals into a process with no outcome tracking will produce better-looking dashboards. It will not produce measurably better decisions, because measurably better may not be something the operation has the infrastructure to determine.

How Does Good Look?

A commercial operation with decision quality infrastructure looks like this: an analyst takes an action on a flagged market and that action is recorded. The outcome—what the market did in the periods following the action—is tracked against what the system expected. The delta between expected and actual is fed back into both the system’s calibration and the analyst’s performance record, not as a punitive measure, but as a development tool. Over time, the system gets better at knowing which actions produce better results, ans so does the analyst.

That loop does not require a complete technology transformation. It requires deliberate architecture, domain expertise in defining what a good decision looks like for a given situation, and organizational commitment to treating decision quality as a first-class metric rather than an afterthought.

The airlines that build it will have something that cannot be bought off a vendor shelf: a commercial operation that learns.

Author’s note: the feedback loop described in this post is a core design principle of the telos Signals platform. The question of whether decisions are good is one that keeps us up at night, and we’re developing a platform that learns.