Google’s TabFM skips training on each dataset and still predicts on unprecedented tables

Most business data is tabular – residing in databases, CRMs, and financial ledgers – yet building a reliable model on it still means training a new one from scratch on every dataset, then maintaining hyperparameter tuning loops, feature engineering, and pipeline retraining to combat data flooding. Google Research suggests a way around that: a new foundational model called TabFM that treats table projection as an in-context learning problem instead.

It can generate predictions for a new, invisible table in one forward pass. For business engineers and AI engineers, this reduces production time from weeks of pipeline engineering to a single API call.

The challenge with traditional ML

To extract reliable predictions from a gradient-optimized tree, data scientists must build and maintain complex data pipelines. They have to clean up dirty inputs, set missing values, encode class variables into numeric formats, and developer custom feature crosshairs.

Once the data is ready, they must use iterative parameter optimization loops, searching for learning values, tree depths, sample rates, and generalization grids to find the best configuration.

Once used, these traditional models "incurring ongoing operational liabilities by monitoring data drift and retraining pipelines to remain accurate," Weihao Kong, Research Scientist at Google Research, told VentureBeat.

Meanwhile, the entire AI industry has moved forward. Generative AI models of text and computer vision have shifted to zero-shot inference, where the model can perform an entirely new task by being told the context.

Large-scale language models (LLMs) already excel at in-context learning, so why can’t we just feed tables into an off-the-shelf LLM?

Because LLMs are trained in natural language rather than structured data, they struggle to process tables directly. First, their context limitations are quickly overcome by medium-sized tables containing just a few thousand rows and hundreds of columns. Second, LLMs suffer from tokenization inefficiencies, misclassifying values ​​and destroying statistical accuracy. Finally, they suffer from structural blindness. If a 2D table is formatted as a 1D text string, LLMs lose track of which value is in which row and column as the table grows.

"That is why, today, it is more efficient to use LLM to write code that handles the engineering feature and call XGBoost than to ask LLM to read the table itself," Kong said.

What is TabFM?

To use TabFM, you do not update any model weights. Instead, you take your historical examples (the training lines with their known labels) and your target lines (the new data you want to predict) and pass them to the model as a single, combined input. The model learns to interpret the relationships between columns and rows directly from this context at runtime.

For example, consider a business analyst trying to predict customer churn. Instead of building a structured data pipeline and training an XGBoost model, they can simply pass a sample of user session data corresponding to a new, active session to TabFM. With one forward pass, the model returns the exchange probability immediately.

TabFM overcomes the limitations of LLMs by treating the data as a grid, preserving its structural integrity without forcing it into a one-dimensional text string.

In order to effectively analyze various table structures while allowing for zero-shot down prediction, TabFM combines the strengths of the previous test structures, TabPFN and TabICL. TabPFN, developed by Prior Labs, first proved that the transformer architecture could perform zero-shot sorting on small tables, although it was difficult to scale computationally on large datasets.

Later, TabICL, developed by France’s National Research Institute of Digital Science and Technology, addressed this limitation by introducing line compression, which allows in-content reading to efficiently process very large tables.

TabFM combines the depth of the TabPFN feature with the active compression of TabICL in a novel hybrid design built in three key ways:

1. Row and column swap: The raw table is first processed with a multi-layer attention module that alternates between both columns (features) and rows (examples). By continuing to travel in both directions, the model naturally captures the interaction of complex factors. This deep content alignment does the heavy lifting that often requires the art of confusing data scientists.

2. Line compression: After this context correction, the information visited across each row is compressed into a single, dense vector representation. TabICL pioneers this by using CLS tokens to compress rich line information into a single vector, "unlike TabPFN v2, v2.5, and v2.6, which run over a full cell grid throughout the network," Kong explained. This greatly reduces the computer footprint.

3. In-Content Learning (ICL): The Causal Transformer then operates on this compressed embedding sequence. This Transformer model uses the TabICL attention method to attend to these dense row vectors, greatly reducing the computational cost and allowing the model to process large data sets efficiently.

TabFM’s biggest selling point is its early training recipe. The model was fully trained on hundreds of millions of synthetic data sets. These datasets are dynamically generated using structural causal models (SCMs) that include many different random functions. Specially trained in synthetic SCMs, TabFM learned the fundamentals of how table elements interact without importing real-world, private CSV files.

TabFM is active

To test the power of the model, Google researchers benchmarked TabFM on TabArena, a comprehensive test environment that includes 51 different tabular data sets in 38 classifications and 13 regression functions.

In these public benchmarks, TabFM’s zero-shoot forecasts are already matching or beating the most targeted baselines. However, Google is careful to note that this does not automatically mean that TabFM will universally rescind best-in-class production models for all business operations.

"Instead of changing advanced production models, the real value of the operational business it unlocks in lean engineering teams is the speed," Kong said. "It allows data analysts and backend engineers to quickly wrap high-quality basic models without a dedicated data science team managing the complex lifecycle."

For high-level operators who want to squeeze more precision, the research team also introduced a "TabFM-Ensemble" to stop. By running the model on 32 different variables and combining the results, TabFM pushes performance even further.

Startups, entrepreneurship, and the future of the cloud

The shift to reading in-context tables presents new trade-offs that engineering teams must consider.

With traditional algorithms, training is slow and expensive, but inference is fast and cheap. TabFM responds to this dynamic. When training time drops to zero, interpretation becomes more difficult. Because the model must process the entire historical dataset as context during each prediction, it requires more computation and memory at runtime.

At this new point, "the training of a common machine learning becomes a ‘filling’ section (KV caching) in the content window," Kong said. Although this pre-fill cost is high, it is only paid once per table, and the cache is reused for all subsequent queries. "Catch prediction delay, which no cache value removes," Kong is added. All new projections need to go through a large transformer. "Any production API that requires single-digit millisecond response times cannot tolerate TabFM’s forward pass."

For developers looking to test the model today, the barrier to entry is low. Google designed TabFM as a replacement for traditional ML workflows, offering a compatible scikit-learn API (TabFMClassifier and TabFMRegressor). It natively handles numeric columns and mixed ranges, works directly with pandas DataFrames, and doesn’t require ordinal encoders or numeric scales. The library supports both JAX and PyTorch backend.

However, business groups need to be aware of current restrictions and licensing restrictions. The architecture model has a hard limit of 10 output classes for classification functions, and is designed for tables with up to 500 elements. More importantly, while Google releases the underlying codebase under the permissive Apache 2.0 license, the pre-trained model weights are published on Hugging Face under the strict tabfm-non-commercial-v1.0 license. Developers can test the model internally, but it cannot be used in commercial products yet.

Looking ahead, Google is dealing with the friction of commercial deployment with its cloud ecosystem. TabFM integrates directly with Google BigQuery, allowing analysts to make intuitive predictions natively with the “AI.PREDICT” command. By placing a predictive base model next to the data warehouse, TabFM can quickly make complex table machine learning as accessible as basic database querying.

In practice, TabFM shines in rapid prototyping, in high-data-flood environments, with small to medium-sized data sets under 100,000 rows. In contrast, teams should stick to traditional models for robust, low-profile APIs, or large tables with more than a million rows, which currently require aggressive row sampling that undermines the competitive advantage of the baseline model.