Google Research introduced TabFMThe foundation model is built to handle tabular data. TabFM is able to perform both classification and regression, without the need for dataset-specific learning. The model makes every prediction from one forward pass. The model frames tabular predictions as in-context problems. This model is currently available at Hugging Face, GitHub.
TL;DR
- TabFM is able to predict the outcome of a game on a table that has not been seen. This requires no tuning, training or engineering.
- In-Context Learning is used to predict the results after reading all of the datasets.
- TabICL is an in-context, row/column learning architecture that combines TabPFN’s style of attention to rows and columns with TabPFN’s style of row/columns.
- The training uses hundreds of millions synthetic datasets derived from structural causal models.
- Google BigQuery is exposing TabFM via an AI.PREDICT SQL Command soon.
What is TabFM?
Tables are the foundation of any enterprise’s data infrastructure. Tables can be used to perform tasks such as detecting fraud and customer churn. Tree-based methods have dominated the space for years. XGBoost and AdaBoost provided robust results for structured data. Google presents TabFM as a tabular version of TimesFM, the zero-shot model for time series.
This reliability came at a price. Fitting XGBoost onto a brand new dataset can be a challenge. .fit() call. Data scientists work for hours optimizing hyperparameters and engineering features. These data scientists do all this to obtain a trustworthy signal out of raw data. TabFM targets this bottleneck.
TabFM follows the logic of large language model zero-shot. LLMs can learn to perform new tasks by using in-context example without changing any weights. The technique used is known as in-context (or ICL). TabFM takes the idea and applies it to tables. In one single pass, it generates all predictions.
It Works
TabFM skips this step entirely. TabFM completely skips this step. TabFM uses the entire dataset in a single, unified prompt. This prompt contains both target rows and training examples. At inference, the model can read column-row relationships.
Text is not a table. The tables are inherently two-dimensional. The meaning of the sentences is not altered by swapping two rows or columns. Instead, standard language models handle one-dimensional ordered sequences. TabFM is a synthesized TabPFN/TabICL hybrid designed to bridge this gap.
The system relies on 3 mechanisms:
- Attention to alternate row and column: A multilayer module of attention processes the raw table. TabPFN switches attention between rows and columns. The contextualization is very detailed and captures all the interactions between features. This tool automates tasks that require manual work.
- Row Compression: The information from each row that is cross-attended compresses to a dense vector.
- In-context learning: These compressed embeddings will be run over by a Transformer. TabICL’s attention to the compressed rows reduces computation costs dramatically. The prediction is still efficient on very large datasets.
Training on synthetic data at scale
Foundation models need vast, diverse data. In the world of open source, it is difficult to find high-quality datasets. The tables in industrial settings contain sensitive and proprietary information. They are therefore not accessible to a wide range of pre-training.
You can generate tables of any size. Google’s researchers call them the best option for this size. TabFM uses hundreds of thousands of synthetic datasets to train. The datasets are dynamically generated using structural causal model (SCMs). The random functions are varied in each. This approach is able to capture the complex relationships and distributions found in actual tables. Researchers report that the model is generalizable to real data not seen before.
Performance Benchmarking
The TabArena team evaluated TabFM. TabArena, a benchmark for live games that calculates Elo from win rates in head-to-head matches, is an example of a living benchmark. This evaluation covers 38 datasets for classification and 13 datasets for regression. Sample sizes vary from 700 up to 150,000.
We tested two configurations. Plain TabFM works out of the box in one forward pass. No tuning or cross validation is required. TabFM-Ensemble includes SVD and cross features. The program computes the optimal weights of a 32 way ensemble by using a solver that does not use negative least squares. This software also uses Platt scale as a calibration for the classification.
TabFM outperforms industry standard algorithms, according to the research team. The GitHub page contains full per-fold statistics and win rates.
| Aspect | Traditional GBDT | TabFM | TabFM-Ensemble |
|---|---|---|---|
| Dataset-specific training | You are Required | No (In-Context Learning) | No, |
| Hyperparameter tuning | Manual | No, | Ensemble weights via NNLS |
| Feature engineering | Manual, domain-specific | You can learn by paying attention | Cross + SVD now available |
| Prediction | Following full training | Single forward pass | 32-way ensemble |
| Calibration | Manual (optional). | — | Platt scaling (classification) |
Get Started with Installation and Code
The repository is cloned and installed locally. This base installation uses CPU-only JAX. The Base Install uses only CPU-based JAX. Cuda Additional pulls the CUDA 12 plug-in and NVIDIA Libraries for GPU runs.
Specific requirements apply to core functions. Python 3.11 is required. You need Python 3.11 or later. jax==0.10.1 The following are some examples of how to get started: flax==0.12.7The modern flax.nnx API. Downloading the weights pre-trained automatically is possible with Hugging Face Hub.
Import numpy as an np
import pandas as pd
Tabfm import tabfm_v1_0_0
TabFMClassifier import from Tabfm
TabFM pre-trained v1.0.0 can be downloaded from the Hugging Face website.
model = tabfm_v1_0_0.load()
# scikit-learn compatible classifier
clf = TabFMClassifier(model=model)
X_train = pd.DataFrame({
"age": [25.0, 45.0, 35.0, 50.0],
"job": ["engineer", "manager", "engineer", "manager"],
"income": [80000, 120000, 90000, 130000]
})
y_train = np.array(["low_risk", "high_risk", "low_risk", "high_risk"])
X_test = pd.DataFrame({
"age": [30.0, 48.0],
"job": ["engineer", "manager"],
"income": [85000, 125000]
})
clf.fit(X_train, y_train)
prediction = clf.predict()
probabilities = clf.predict_proba(X_test)
print("Predictions:", predictions)
print("Class Probabilities:n", probabilities)
You can find out more about this by clicking here. You can also fit() It prepares numerical and ordinal scalers. The model does not learn weights from your data. This pattern is mirrored by the regressor. TabFMRegressor The following are some examples of how to get started: reg.predict().
Use Cases with Examples
APIs can be used to perform common prediction tasks. In the context of customer retention, past customers are labeled as retained or churned. TabFM measures churn risks for new clients in a single pass.
Age, employment, and income are all factors that can be used to assess credit risk. Labels indicate low_risk The following are some examples of how to use high_riskThe sample code is an example. The new applicant is scored immediately without any training.
House price predictions are a good fit for regression. The context rows include square footage, neighborhood and other information. TabFM provides a price prediction for listings that have not yet been seen.
Interactive Explainer
state.rows.forEach(function(row,ri){
bodyHtml += “
The cols().forEach(function(c){
BodyHtml = “
“;
});
Target cell
if(t.target.numeric){
The bodyHtml code is “If you want to know more about, click here.
“;
} else {
Sel is equal to ““;
” bodyHtml
“;
}
BodyHtml = “
“;
bodyHtml += “
“;
});
bodyHtml += “
“;
tbl.innerHTML = head + bodyHtml;
tbl.querySelectorAll(“input,select”).forEach(function(el){
el.addEventListener(“change”, function(){
var ri = +el.getAttribute(“data-ri”), k = el.getAttribute(“data-k”);
var v = el.value;
var numeric = (el.type === “number”);
state.rows[ri][k] = numeric ? parseFloat(v) : v;
});
});
tbl.querySelectorAll(“.del”).forEach(function(b){
b.addEventListener(“click”, function(){
if(state.rows.length=10;
}
function cellInput(c,val,ri){
if(c.opts){
var s = ““;
}
return ““;
}
function renderTestInputs(){
var t = TASKS[state.task], wrap = $(“testInputs”);
var html = “”;
cols().forEach(function(c){
html += “
“;
if(c.opts){
html += ““;
} else {
html += ““;
}
html += “
“;
});
wrap.innerHTML = html;
wrap.querySelectorAll(“input,select”).forEach(function(el){
el.addEventListener(“change”, function(){
var k = el.getAttribute(“data-tk”);
state.test[k] = (el.type===”number”) ? parseFloat(el.value) : el.value;
});
});
}
// —- illustrative predictor: distance-weighted k-NN over context rows —-
function ranges(){
var t = TASKS[state.task], r = {};
t.numeric.forEach(function(c){
var vals = state.rows.map(function(row){return row[c.key];});
var mn = Math.min.apply(null,vals), mx = Math.max.apply(null,vals);
r[c.key] = (mx-mn) || 1;
});
return r;
}
function distance(a,b,rg){
var t = TASKS[state.task], d = 0;
t.numeric.forEach(function(c){
var diff = (a[c.key]-b[c.key])/rg[c.key];
d += diff*diff;
});
t.cat.forEach(function(c){
if(a[c.key]!==b[c.key]) d += 1;
});
return Math.sqrt(d);
}
function predict(){
var t = TASKS[state.task], rg = ranges();
var scored = state.rows.map(function(row){
var dist = distance(state.test,row,rg);
return {row:row, dist:dist, w:1/(dist*dist+0.05)};
}).sort(function(a,b){return a.dist-b.dist;});
var k = Math.min(5, scored.length);
var top = scored.slice(0,k);
var wsum = top.reduce(function(s,x){return s+x.w;},0);
if(t.target.numeric){
var est = top.reduce(function(s,x){return s + x.w*x.row[t.target.key];},0)/wsum;
showRegression(est, top);
} else {
var probs = {};
t.target.opts.forEach(function(o){ probs[o]=0; });
top.forEach(function(x){ probs[x.row[t.target.key]] += x.w; });
t.target.opts.forEach(function(o){ probs[o] = probs[o]/wsum; });
showClassification(probs, top);
}
}
function fmtMoney(v){ return “$”+Math.round(v).toLocaleString(“en-US”); }
function neighborLine(top){
var t = TASKS[state.task];
var parts = top.map(function(x){
var lab = t.target.numeric ? fmtMoney(x.row[t.target.key]) : x.row[t.target.key];
return “"+lab+"“;
});
return “Nearest context rows used: “+parts.join(” “);
}
function showClassification(probs, top){
var t = TASKS[state.task];
var best = t.target.opts[0];
t.target.opts.forEach(function(o){ if(probs[o]>probs[best]) best=o; });
var isChurn = (best===”Yes”);
var conf = (probs[best]*100).toFixed(1);
var bars = “”;
t.target.opts.forEach(function(o){
var pct = (probs[o]*100);
var color = (o===”Yes”) ? “var(–neg)” : “var(–pos)”;
bars += “
“+
““+
“
“;
});
$(“result”).innerHTML =
“
Prediction · class probabilities
“+
“
Churn: “+best+”
“+
“
Confidence “+conf+”% · decided purely from the context rows above.
“+
“
“+bars+”
“+
“
“+neighborLine(top)+”
“+
“
In TabFM, this maps to predict_proba(). No weights were updated — “+
“the answer is read from the context in a single forward pass.
“;
reveal();
}
function showRegression(est, top){
$(“result”).innerHTML =
“
Prediction · estimated value
“+
“
“+fmtMoney(est)+”
“+
“
Estimated from the “+top.length+” nearest context rows, weighted by similarity.
“+
“
“+neighborLine(top)+”
“+
“
In TabFM, this maps to reg.predict(). No per-dataset training ran — “+
“the estimate comes from the context in one forward pass.
“;
reveal();
}
function reveal(){
var r = $(“result”);
r.className = “result show”;
setTimeout(function(){
r.querySelectorAll(“.bar-fill”).forEach(function(f){
f.style.width = f.getAttribute(“data-pct”)+”%”;
});
},40);
$(“runNote”).textContent = “Try editing the context, then predict again.”;
postSize();
}
// —- wiring —-
$(“tabs”).addEventListener(“click”Function(e){
var b = e.target.closest(“.tab”); if(!b) return;
$(“tabs”).querySelectorAll(“.tab”).forEach(function(x){x.classList.remove(“on”);});
b.classList.add(“on”);
loadTask(b.getAttribute(“data-task”));
postSize();
});
$(“addRow”).addEventListener(“click”Function());
$(“resetRows”).addEventListener(“click”FunctionIf you want to know more about, click here.Function neighborLine (top)var top.map() = partsRegression function (left, right)(){
state.rows = clone(TASKS[state.task].rows);
renderTable();
postSize();
});
$(“runBtn”).addEventListener(“click”, predict);
—- Auto-resize WordPress iframe embed —-
Size of function posts(){
var h = document.body.scrollHeight;
if(window.parent){ window.parent.postMessage({tabfmHeight:h}, “*”); }
}
window.addEventListener(“load”, postSize);
window.addEventListener(“resize”, postSize);
var ro = window.ResizeObserver ? new ResizeObserver(postSize) : null;
if(ro) ro.observe(document.body);
loadTask(“churn”);
setTimeout(postSize, 60);
})();

