Source: Unite.AI
NVIDIA, working with IIT Madras and its AI4Bharat lab, on October 16, 2026 introduced Nemotron-pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.8 trillion tokens across 22 Indian languages that is releasing soon, together with the open, GPU-accelerated data generation pipeline used to create it.
The 22 languages are those listed in India’s Constitution, which the announcement describes as the first languages for 97% of Indians, nearly 1.2 billion citizens at the 2011 Census. Authors on the post span AI4Bharat researchers Mohammed Safi Ur Rahman Khan, Mitesh Khapra, and Anoop Kunchukuttan and NVIDIA’s Niket Agarwal, Raviraj Joshi, Anusha Kamath, Markus Kliegl, Yev Meyer, Sriharsha Niverty, Kiran Praveen, and Utkarsh Vaidya. Khapra, who heads AI4Bharat, has named the core problem behind Indian language technology trailing English: “we do not have enough data for Indian languages.”
The pipeline captures the full generation recipe, including prompts, LLM workflows, and quality checks, so developers can adapt it to their own source material. NVIDIA said every record traces back to a curated source document and each output is checked against its input.
Corpus Composition and Language Coverage
The corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer. NVIDIA notes the counts describe the retained corpus, including accepted rewrites, rather than a training run’s token budget, and that the counts depend on the tokenizer used.
Across 34 language-script configurations, the corpus provides native-script data for all 22 languages and romanized text for twelve. Twelve languages have both native-script and romanized data: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, and Urdu. Ten have native-script data only: Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali, and Sindhi.
The post defines native-script as the writing system, with the text still translated from English, while romanized means Indic-language text written in Latin script. For each of the twelve dual-script languages, romanization covers a subset of the accepted native-script translations.
Two Translation Workflows
Generation starts from documents in 15 Nemotron-CC-v2 subsets covering web text, question-answer material, mathematics, code, and other specialized content. NeMo Data Designer, NVIDIA’s open-source synthetic-data framework, runs translation, quality checks, and targeted repair, with source identifiers retained throughout so an output can be traced to its original document.
For the twelve dual-script languages, documents are translated in full using Google’s Gemma-4-31B-it, which the team said it selected by comparing models on complete documents. Prompts distinguish prose to translate from content to preserve, including code, equations, URLs, and formatting. The same model then assesses each output against its English source using a rubric inspired by the Multidimensional Quality Metrics framework, covering meaning, fluency, terminology, style, and formatting. Only outputs scoring at least 75 out of 100 are retained; lower-scoring outputs are repaired from the assessment feedback and reassessed against the same threshold. NVIDIA reports this repair loop recovered 438.4 billion tokens across native-script and romanized data, already included in the corpus total, and that sampled human reviews plus tests with deliberately introduced errors informed improvements to prompts and checks.
Accepted native-script translations provide the starting point for romanized data through a separate script-conversion step. According to NVIDIA, an initial human comparison found this two-step approach produced fewer hallucinated additions and less residual English than generating romanized text directly from English.
For the other ten languages, translation uses Indic Translate, a document-level model for English and 22 Indian languages from Bodhan AI, a nonprofit center for AI in education incubated at IIT Madras, with Gemma-4-31B-it assessing each translation against its English source. Eight languages keep the whole-document approach; longer Santali documents are translated in overlapping windows and reconstructed, and Manipuri is translated into Bengali script before conversion to Meitei Mayek, its native script. These workflows keep the 75-out-of-100 threshold and produce native-script data only, without feedback-based rewriting.
Continued Pretraining Results
The Nemotron 3.5 Super VL technical report compares the checkpoint immediately before Indic continued pretraining with the checkpoint after that stage. The configured budget was 100 billion training tokens, with 10 percent (10 billion tokens) allocated to a mixture of synthetic and natural Indic text and the remaining 90 percent maintaining the preceding training mixture’s relative proportions. Scores increased on all five reported Indic knowledge and reasoning benchmarks: MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy, and Indic GSM8K, with gains ranging from 4.98 to 9.82 percentage points. NVIDIA notes the comparison covers a training stage combining synthetic and natural Indic data rather than isolating the synthetic corpus’s contribution, and that GSM8K excludes Odia because evaluation exceeded the serving context limit.
Against Nemotron 3 Super, the released Nemotron 3.5 Super VL scores higher on five of six task benchmarks shown, spanning knowledge, reasoning, mathematics, code, and translation, while instruction following on IFEval 1.5 dipped 2.04 points, according to the post. It also improves across all four dimensions of IndiVibe, Sarvam AI’s open benchmark of assistant-style prompts in Indian languages: usefulness, fluency, language and script fidelity, and appropriate verbosity.
NVIDIA said it plans to build on this corpus in future Nemotron models, and that evaluating the full corpus, including recovered rewrites and the ten-language group, remains a next step for controlled studies.
Availability and Stated Limitations
Dataset configurations use language, script, and region tags, with hi-Deva-IN denoting Hindi in Devanagari and hi-Latn-IN denoting romanized Hindi, and Hugging Face Datasets streaming lets readers inspect records without downloading the entire corpus. The open-source Nemotron Bharat translation plugin for NeMo Data Designer provides translation and evaluation prompts, optional feedback-based rewriting, document windowing, and script-remapping support.
NVIDIA cautions that translation does not replace text originally written in Indian languages or guarantee coverage of local knowledge and culture, and it recommends combining the corpus with appropriately sourced natural Indic text. Quality assessment is primarily automated, complemented by sampled human review whose coverage varies across languages, and in the twelve-language workflow the translator also serves as the judge. The post states that retained text can still contain errors, that the ten-language workflow needs broader native-speaker validation, and that source-ID checks remove processing duplicates without performing semantic deduplication of similar content.
Companion Benchmark Suite
Alongside the corpus, AI4Bharat at IIT Madras, with support from NVIDIA, introduced Indic-LLM-Eval-Suite, an open and evolving collection of offline benchmarks for evaluating LLMs across Indian languages.
The first version assembles existing Indic benchmarks, refreshed versions including MILU-1.5 and IndicIFEval-1.5, and adapted English benchmarks across eight capability categories: general chat, instruction following, mathematics, STEM, commonsense reasoning, knowledge, code, and translation. The evaluations are integrated with NVIDIA NeMo-Gym, providing a common evaluation interface while retaining benchmark-specific scoring procedures.
AI4Bharat reports that its audit of MILU found incomplete questions, missing figures, and incorrect translations, which were addressed in the revised MILU-1.5, and that it fixed issues in IndicIFEval’s evaluation code where certain constraints were incorrectly checked or silently ignored. The suite’s construction methodology pairs benchmark-aware translation with automatic quality assessment, targeted correction, and native-speaker verification.
AI4Bharat said it evaluated several contemporary open-weight Indic and multilingual LLMs across the suite and found that no single model consistently performs best across all tasks. The lab describes the initial release as a starting point and says it plans to add benchmarks, incorporate community contributions, and periodically refresh results to track the progress of Indic LLMs over time.