source: google research: introducing tabfm: a zero-shot foundation model for tabular data
level: technical
tabular data is everywhere in business, from predicting customer churn to spotting fraud. for years, tree-based models like xgboost have been the go-to tools, but they need a lot of manual work. data scientists spend hours on hyperparameter tuning and feature engineering for each new dataset. this slows down the process of getting useful predictions.
tabfm changes this by treating tabular prediction as an in-context learning problem. instead of training a model for each task, you give tabfm the whole dataset—training examples and test rows—as a single prompt. the model figures out the relationships between columns and rows on the fly. its architecture uses alternating attention across rows and columns, then compresses each row into a dense vector. a final transformer step makes predictions efficient, even for larger tables.
training tabfm required a lot of data, but real-world tabular datasets are often private or scarce. so the model was trained entirely on hundreds of millions of synthetic datasets. these were created using structural causal models with random functions to mimic real data variety. in tests on the tabarena benchmark, tabfm outperformed heavily tuned traditional models. an ensemble version with extra features and calibration pushed results even higher. tabfm will soon be available in google bigquery via a simple sql command.
why it matters: it removes the need for manual model tuning and feature engineering, letting data scientists get accurate predictions on new tabular data instantly.
source: google research: introducing tabfm: a zero-shot foundation model for tabular data