Skip to content

Amazon Mitra-v2 — Tabular Foundation Models and Fine-Tuning

Last reviewed: 19 September 2026

Édition française : Amazon Mitra-v2 — Modèles de fondation tabulaires et fine-tuning

Tabular foundation models come in two families: those that answer straight away (zero-shot, such as Google's TabFM) and those you specialise on your own data in a few minutes. Mitra-v2, released by Amazon, belongs to the second: 77 million parameters, trained only on synthetic data, and designed for fine-tuning. This article explains how it works, what it brings compared with a classical data-science project, and where it stands against TabFM — with the same Senegalese use case as the companion article: the solar production of a house in Badoudou, near Sokone.

What you will understand

  • What a mixed synthetic prior is, and why it is the heart of Mitra.
  • How a regression can be treated as a classification over 1,000 bins, and why that gives forecasts with uncertainty.
  • The difference between a generalist model (zero-shot) and a specialist model (fine-tuning).
  • What the public benchmarks say, and how to read a model comparison without being fooled.
  • When fine-tuning is worth it… and when it is not.

Prerequisite: having read, or skimmed, the Google TabFM article, which lays out the basic notions (in-context learning, foundation model, test protocol).

1. Recap: why tabular models need a “prior”

A generalist foundation model learns on millions of synthetic tasks. The quality of those tasks — their prior distribution, their prior — determines what the model will be able to do later on your real data.

Two strategies emerge:

  • Betting on extreme variety: generating highly diverse tables and relying on in-context learning to adapt to anything, without ever changing the weights. That is the TabPFN / TabFM path.
  • Betting on diversity plus specialisation: generating mixed synthetic priors (several families of generators, including a hybrid causal prior), then letting the user adjust a few fine-tuning steps on their own task. That is the Mitra path.

Mitra-v1 (“Mixed synthetic priors for enhancing tabular foundation models”, 2025) showed that more diverse priors produce a model that specialises better and faster. Mitra-v2 pushes the logic further: a much broader pre-training distribution, a context ten times longer, three times as many variables.

2. Architecture: a compact 2D Transformer

Mitra-v2 architecture

The key points, without unnecessary jargon:

  • 12 layers, 76.7 million parameters. For comparison, TabFM announces 1.6 billion parameters: Mitra-v2 aims for performance at a fraction of the inference cost.
  • “2D” attention. The model attends to rows and columns at the same time: a row is an example; a column is a variable. Many tabular architectures process those two directions one after the other; here information flows in both directions at every layer.
  • Long context. The training rows are passed to the model as context (in-context learning), ten times more rows than in version 1 — that is what makes it possible to work on real datasets, not only on toys.
  • A distributional head with 1,000 bins. This is the most elegant idea: for a regression, instead of predicting a value directly, Mitra-v2 splits the target into 1,000 bins and predicts a probability for each one. The point forecast is the mean of that distribution.

The practical consequence: you do not get only “14.2 kWh”, but a distribution — hence quantiles, confidence intervals, and the ability to say “I have an 80% chance of producing more than 11 kWh tomorrow”. For solar sizing, that is exactly what you need: you do not size a battery on an average, but on the bad days.

3. Generalist or specialist: two ways to use a foundation model

In-context learning versus gradient descent

Generalist (zero-shot) Specialist (fine-tuning)
Examples TabFM, TabPFN Mitra-v2
What changes during use nothing: the weights are frozen the weights are adjusted on your data
Effort no tuning a training recipe to run
Hardware CPU is enough for a small dataset GPU required (the published recipe uses a GPU)
Expected gain very good immediately the last few points of performance, on a specific domain
Main risk licence and the limits of the domain seen during training overfitting when data is scarce

The published reference recipe for Mitra-v2 is 50 fine-tuning steps with 8 “bagged” copies (8-fold bagging): the model is fitted eight times on subsamples, and the eight predictions are averaged. That reduces variance — the main enemy when you have little data.

# Inference / fine-tuning environment for Mitra-v2
uv pip install "autogluon.tabular[mitra]>=1.6" "tabarena>=0.1.0"

[!NOTE] Licence. Mitra-v2 is released under Apache-2.0: the weights, the inference code, the fine-tuning code and the evaluation results. This is a notable difference from weights under a non-commercial licence: here, production use is permitted without prior negotiation.

4. What the benchmarks say (and how to read them)

Mitra-v2 is evaluated on TabArena and TALENT, two benchmark suites bringing together more than 300 real datasets (credit risk, clinical prediction, equipment failure detection, real-estate price estimation…), under two evaluation protocols.

The results announced by the authors:

  • performance in the region of TabFM and EXAONE Tabular (the best models in the category), ahead of TabPFN-3 on TabArena, in both classification and regression;
  • 5% of TabFM's size (77 M versus 1.6 B parameters) for comparable performance;
  • on TALENT, among the leaders, ahead of TabPFN-3 and TabICLv2;
  • first on classifications with more than 10 classes, even though the model was pre-trained only on tasks with at most ten classes — a sign that structure learning transfers.

Which model for which project

Three reading precautions, valid for any model comparison:

  1. A ranking is not a guarantee for your problem. Benchmarks aggregate dozens of tasks: the average can be excellent while your particular case is poor.
  2. The gap between the top three is measured in points. Choosing between TabFM, Mitra-v2 and TabPFN-3 on a leaderboard often amounts to choosing on the licence, the inference cost and the ecosystem (here: AutoGluon, very well tooled, and Apache-2.0).
  3. Always compare against a simple baseline — mean, linear regression, boosted tree — on your data, with your train/test split.

5. The use case: forecasting the solar output of a house in Badoudou (Sokone)

The dataset is described in detail in the companion article; the essentials:

Use-case architecture

  • Photovoltaic production: real data. NASA POWER provides 8,760 hours (2025) of irradiation, temperature and humidity for Sokone; PVGIS gives the monthly yield per kWp for 7 regions of Senegal. Hourly production is computed by a physical model (performance ratio 0.78, thermal de-rating −0.4%/°C).
  • Consumption: a documented appliance model. Refrigerator, fans, lighting, TV, chargers, pump: ≈ 3.8 kWh/day, that is 1,378 kWh per year.
  • Question: is 1.5 kWp + a 5 kWh battery enough, and what is left to buy from the grid?

The full hourly simulation gives:

Configuration Self-consumption Grid purchase Annual surplus
1.5 kWp, no battery 41.3% 809 kWh/year 1,579 kWh/year
1.5 kWp + 5 kWh battery 97.0% 1 kWh/year 728 kWh/year
3.0 kWp + 5 kWh battery 97.3% 0.5 kWh/year 2,880 kWh/year

With the SENELEC tariffs of January 2026 (82 FCFA/kWh for the first 150 kWh per month), covering 1,378 kWh/year represents about 113,000 FCFA of avoided purchases per year — the details of the assumptions and the full calculation are in the TabFM article, section 6.6.

Where Mitra-v2 brings something specific here: forecasting production is a regression, and sizing an installation for a bad day is the real issue. A point forecast does not say much; a distribution (low quantiles, median, high) says how many days a year the battery will be empty. That is precisely what the head with 1,000 bins provides, and what a simple boosted tree does not give without extra work (quantile regression, bootstrap, conformal prediction).

6. Should you fine-tune? The honest calculation

Fine-tune if:

  • you have several thousand rows and a GPU;
  • your domain is particular (specific sensors, business units, local seasonalities);
  • you need the best possible accuracy and a well-calibrated distribution;
  • you can repeat the operation at every data update (industrialisation).

Do not fine-tune if:

  • you have a few dozen or a few hundred rows: zero-shot already does the job, and 50 fine-tuning steps on 84 rows run the risk of learning mostly noise;
  • you have no GPU available: zero-shot inference runs on CPU;
  • your problem is unstable over time (the specialised model will have to be rebuilt often);
  • you are looking first for a fast answer: zero-shot gives 80% of the result in 20% of the time.

In all cases, the winning approach is a ladder: naive baseline → classical model → zero-shot foundation model → fine-tuning. You only move up a rung if the previous rung is not enough.

7. Limits and pitfalls

  • Everything is synthetic. Mitra-v2 has never seen real data during training: domains very far from the priors remain fragile. The model card explicitly invites you to evaluate on representative validation data before any use.
  • Fine-tuning = risk of overfitting when data is scarce — hence the 8 bagged copies of the official recipe, which must not be “simplified”.
  • It is not a causality tool. The model predicts one variable from others very well; it does not say why, nor what would happen if an input changed.
  • Classic mistake: judging on the average. With a distributional head, you can be excellent in probability and average on the mean value. Choose the metric that matches the decision (the cost of a day without battery, not RMSE).
  • The hidden cost is industrialisation: versioning the model and the training context, monitoring data drift, and documenting what was fine-tuned and when.

8. Key takeaways

  • Mitra-v2 is a tabular foundation model with 77 M parameters, pre-trained only on mixed synthetic priors, designed to be fine-tuned: it reaches the level of the best models in its category with a fraction of their size.
  • It treats regression as a classification over 1,000 bins: you get a distribution, hence uncertainty — valuable for sizing a solar installation.
  • Everything is released under Apache-2.0 (weights, inference, fine-tuning, evaluations), which removes the licence question that complicates commercial use of some competing weights.
  • On the Senegalese case, most of the result comes not from the model but from the protocol: real data, an explicit physical model, an honest test split, a naive baseline to beat.

Resources