There are important differences between the AI that is just working today, and the AI that stays at scale. Many companies prepare hard for the first one without asking if they are building the second one.
Speed without discipline and strategic direction is a liability, not an asset. The hardest part of building AI at scale isn’t getting the model to work once. Building systems continues to work, goes beyond individual teams and use cases, and evolves consistently over time.
Today’s AI systems do more than just predict and optimize. They talk, consult, and increasingly take action. An autonomous system that makes decisions on behalf of the traveler creates a very different set of expectations regarding reliability, governance, and accountability. As AI takes over many of those roles, the principles of how these systems work are more important than ever.
We’ve spent years applying AI and machine learning (ML) to the entire traveler journey – from personalization, ranking, and recommendations, to fraud prevention, customer support, and, most recently, AI-driven experiences with agencies. That depth of knowledge is what led us to develop a set of ML and AI principles to guide how we build, deploy, and transform AI systems across our company.
The goal is simple: Make sure the systems we build create real business value, scale, and operate securely. These principles describe how we measure, design, manage, and operate our systems.
From principles to practice
Publishing principles are the easy part. The most difficult and important task is to implement the work methods: Recommendations, requirements, tools, and release processes used by the teams.
We have started using ‘Agenttic Release’ tollgates: A set of recommended and, in some cases, required tests before introducing agent AI features. These tollgates translate principles such as clear ownership, risk-based governance, audits, safe releases, and monitoring into team expectations.
Some of these recommendations and requirements are already automated and integrated into the software life cycle (SDLC). Over time, the goal is for these expectations to become central to how we design, analyze, validate, implement, and monitor AI systems from scratch.
Results: Measuring what really matters
The first test of any model is whether it improves the business outcome, and ultimately, the traveler experience — not whether it improves a technical metric.
-
Align models and metrics with business impact: All ML efforts should be directly aligned with a key business outcome or traveler experience metric. Technology development is useful intermediate points, not end goals.
-
Prepare for reimbursement: The value the model creates must justify the cost to develop, train, and monitor, as well as the operational complexity it adds. Popular solutions that deliver a lasting impact compared to what they cost to implement.
-
Validate complexity against a solid foundation: Difficulty should be earned, not thought. Start with a solid foundation: A common existing model, a simple heuristic, an off-the-shelf solution. Access to special models or complex structures only when simple options cannot meet the bar.
-
It requires both online and offline testing: No model goes to widespread use in offline validation alone or jumps straight to A/B testing. Every model should work for offline and online analysis. Over time, our offline observations should reliably predict what we see online.
Design: building systems that transcend the teams that build them
Getting the model to work is one challenge. Making its value extend beyond one group or use case is very difficult.
-
Build on shared foundations; work especially where you have to: Shared, platform-wide foundations for core competencies, data representations, and model building blocks. Expertise should build on those foundations, not stack isolated stacks, so when the foundation improves, the benefits flow throughout the organization.
-
Treat data like a first-class product: The quality of a model is bound by the quality of its data. We need to maintain strong pipelines, clear lineage, reproducibility, and reusable features built with documented ownership, clear schemas, and SLAs that other teams can rely on.
-
Prioritize general rather than local settings: If two methods work in the same way, choose the one that can learn, material, and work patterns that can be reused across teams, brands, and use cases. We must prepare not only for local operations, but also how quickly improvements can spread throughout the company and be integrated over time.
-
Reduce and set the rules of the craft business: Manual rules are sometimes necessary for policy, security, or compliance, but they should be transparent and updated regularly, never introducing weak models or a source of permanent maintenance debt.
-
Automatic reproducibility and traceability: Training data, features, configurations, test results, deployment versions, and key decisions should all be documented and retrievable. That’s what allows you to fix a production problem months later and give ownership without losing the facility’s information.
Trust: ownership, governance, and working responsibly at scale
Post bar AI is not just "does it work?" That’s right "can we stand behind it?" Trust is not something you add at the end; it is earned over time and maintained throughout the life cycle of every model we ship.
-
Provide clear ownership and accountability: Every model needs defined ownership throughout its lifecycle – business owner, product owner, AI owner, and functional owner. These do not have to be four people, but the ties should be clear. Who will be accountable for the results? Who is responsible if the model drifts? Who responds to an incident at 2 am? Without this, the models become orphans and problems arise without their owner.
-
Adhere to standards and governance: AI and ML models must use approved platforms and comply with established company standards, release gateways, and governance processes. Operating outside of these safeguards requires a clear, defined method of correction or withdrawal, rather than an open exception.
-
Control equally at risk: The review rate, test rigor, and human oversight should be proportional to the impact of the model. A customer-facing model that affects pricing or availability for millions of travelers requires a much higher bar than an internal tool used by a small team. In high impact, security-sensitive systems, or highly autonomous, human-accessible checkpoints are built from scratch.
-
Design for fairness, privacy, and transparency: We continuously check for unintended biases, have strong data guidelines, and want to be transparent when decisions benefit users. This is compiled from scratch, not added.
-
Design for safe discharge, recovery, and control: The deployment is in progress, there are undo methods, reverse methods, and circuit breakers ready before launch. The ability to safely reverse a shipment is just as important as the ability to ship it.
-
Continuously monitor and adapt: Once live, teams must actively monitor quality, drift, latency, cost, and business performance and retrain or rescale when data changes. The team should always be able to explain how their model works now, not just how it performed when it was launched.
These principles do more than define how we build. They explain what we intend to deliver and how we stand behind it. In a world where AI systems are becoming increasingly influential and making real decisions for real travelers and partners, these standards matter. When used consistently, they create a responsible AI that lasts.
Xavi Amatriain is Chief AI and Data Officer at Expedia Group
Xavier will share more details about Expedia properties during his session at VB switch July 14 at 11:10 am PT. You will discuss: "Expedia’s plan is to build independent agents and state-of-the-art transaction systems."
Interested in attending VB Transform 2026? Register here. A select number of complimentary passes are also available to senior technology leaders. contact us to find yours.
