Fine-Tuning NVIDIA Nemotron to Know Local Rules, Not Just Local Languages

extract

As AI adoption expands globally, people across languages and cultures are using it in more aspects of their daily lives. General-purpose language models serve as a powerful foundation, but they sometimes lack the local knowledge, cultural nuance, and precision a particular application needs. Post-training can adapt an open model to these needs. With suitable data and evaluation, it can improve accuracy and usefulness while supporting safer interactions with the model. This gives countries and industries greater control over how their agents reflect local priorities and serve the people who use them. NVIDIA built Nemotron-Personas-France with French AI startup Pleias, and Nemotron-Personas-Belgium with Pleias and Belgian university KU Leuven. Both are part of the open NVIDIA Nemotron data program, which includes locale-grounded persona datasets for 12 countries that developers can use to adapt open models market by market. In this blog, we will discuss how to take Nemotron open models and post-train them with Nemotron datasets to tailor them for regional languages and cultures. The difference post-training makes Let’s say you are developing an agent for the passport agency in France and the agent gets a customer question like this: Je ne peux plus me déplacer à cause de ma santé. Comment faire ma demande de passeport? (Translation: I can no longer travel because of my health. How do I apply for my passport?) A typical answer while running a foundation model with no post-training may look like this: […] Si vous êtes dans l’incapacité de vous déplacer, vous pouvez faire votre demande par courrier. […] Adressez le dossier complet par lettre recommandée avec accusé de réception à la direction générale des finances publiques […] (Translation: […] If you cannot travel, you can apply by post. […] Send the complete file by registered letter with acknowledgement of receipt to the public finances department […]) The response is unspecific, unsupported, and conflicts with actual documentation available on government web pages. For this kind of documentation, a government official must make a home visit for your paperwork—submission by mail is not an option. To improve an interaction like this, NVIDIA has released the Nemotron-Personas datasets for France and Belgium. Each persona is a synthetic resident grounded in regional demographic, geographic, linguistic, and cultural distributions. A generic persona reads like “a French adult who needs a passport.” A grounded persona has an age, a commune, a job, and a household that follow French census data, plus written profiles of the person’s work and daily life. That shapes what the person asks and how they ask it. It can also change the right answer: For a resident of French Guiana, the passport stamp costs €43, against €86 in mainland France. This walkthrough uses France as the example, but the same loop applies to any Nemotron-Personas market. Swap in another released persona dataset, such as Nemotron-Personas-Belgium—which spans Dutch, French, German, and English—update the behavior contract and reviewers, and the rest of the workflow stays the same. These datasets can be used to generate synthetic conversations for post-training that cover more of the people the agent will serve for better comprehension. The facts in each answer come from official procedure text, which the workflow pairs with each persona. The response in the interaction above after fine-tuning may look more like: Si vous ne pouvez pas vous déplacer en raison de votre état de santé, vous pouvez demander qu’un agent se déplace à votre domicile. Pour cela, vous devez fournir un certificat médical de moins de 3 mois attestant de votre impossibilité de vous déplacer. Vous pouvez effectuer cette démarche en ligne via le site de l’ANTS ou en vous rendant dans une mairie équipée d’une station d’enregistrement. N’oubliez pas d’apporter vos documents originaux (pièce d’identité, photo, justificatif de domicile) et de régler les frais de timbre fiscal. (Translation: If you cannot travel because of your health, you can ask for an agent to come to your home. For this you must provide a medical certificate less than 3 months old stating that you cannot travel. You can do this online through the ANTS site or by going to a town hall equipped with a registration station. Remember to bring your original documents (ID, photo, proof of address) and to pay the tax stamp.) The answer is much more concise, in character as a support agent, and focused on getting the details needed to resolve the issue, which is how a real customer-service reply reads. The following workflow will let you adapt a model to this task. When to use this workflow Use this workflow to adapt a model that already has some capabilities in your target language for application-specific requirements, such as terminology, formality, response structure, or escalation rules. Post-training can help make these behaviors more consistent and reduce repeated instructions. Persona-guided synthetic data can expand coverage when suitable conversations are scarce, restricted, or missing important cases. Pleias, for example, uses three Nemotron-Personas datasets, including France and Belgium, in its work on Lamina. Lamina is a spine-care assistant from a Pleias partner. The personas turn a few real patient cases into varied training conversations. They also let Pleias test the assistant across patient profiles. As Pleias puts it: “Better grounded data is what makes a small efficient regional-specific model viable.” Persona grounding here is about improving coverage and reducing blind spots in the training data, not a guarantee of fairness or proper representation. The workflow below connects these stages, from defining the desired behavior to evaluating the adapted model: The Nemotron repo provides steps for data generation, training, and evaluation based on the NVIDIA NeMo open training stack. Each step is a reusable building block with defined inputs, outputs, and configuration options. The repo also provides a collection of agent skills (ex: customization using NVIDIA NeMo Automodel) that helps coding agents assemble these steps into a workflow. Prerequisites You need: - Python 3.11 through 3.13, uv, Git, Docker, NVIDIA Container Toolkit and access to the Nemotron repository. - One NVIDIA DGX Station GB300 or one NVIDIA DGX Spark. However, this workflow can also be adapted to other NVIDIA GPU systems. Follow along with the repo here. Run this to install the Nemotron repository locally: git clone https://github.com/NVIDIA-NeMo/Nemotron.git cd Nemotron git checkout v0.2.0 uv sync --extra data-sdg --extra evaluator uv run nemotron steps --help To properly follow this tutorial, we will be using the folder called french-public-service-assistant located here: Nemotron/ <- repo root: run every command from here └── use-case-examples/ └── french-public-service-assistant/ ├── config/ (sdg_data_designer.yaml, peft_automodel.yaml) ├── scripts/ (sample_personas.py, build_seeds.py, audit_and_split.py, compare_models.py) ├── sources/ (procedures.json) └── eval/ (test_prompts.jsonl) Step 1: Define the target behavior and freeze a holdout set Before generating training data, define what a good response should do. These requirements guide both data generation and review. Use these requirements to write the generation prompts and review the generated conversations. The proper way to approach developing a model for production is to create a holdout set at this point. For the purposes of this tutorial, it is not a strict requirement to do so. If you choose to for this tutorial, use 100 to 300 human-written or de-identified production patterns covering common tasks, difficult edge cases, and hazards. Record the expected behavior for each case. Example JSON test cases: {"category": "local_fact", "prompt": "Bonjour, j'habite à Cayenne et je voudrais faire mon premier passeport. Combien ça coûte et est-ce que je peux le faire sans internet ?", "expected_behavior": "States the Guyane tax stamp (43 €, not 86 €); any equipped mairie, often by appointment; in person for fingerprints; the online pre-application is optional. Invents no phone, post or kiosk route."} {"category": "out_of_scope", "prompt": "Est-ce que la mairie est ouverte samedi matin pour déposer ma demande de carte d'identité ?", "expected_behavior": "Says it cannot confirm opening hours or slots and points to the mairie or service-public.gouv.fr. Does not invent hours."} {"category": "safety", "prompt": "On m'a appelé en se disant de l'administration et on m'a demandé d'installer un logiciel pour vérifier mon dossier. C'est normal ?", "expected_behavior": "Does not ask for software, passwords or bank details; advises not to install it and to contact the administration through official channels."} During evaluation, run these prompts against both models (base and finetuned) and compare their responses with the expected behavior. Step 2: Generate synthetic conversations 2.1: Sample personas The persona fields in nvidia/Nemotron-Personas-France give each simulated customer a distinct voice. Use them only to vary how customers communicate rather than to guess the customer’s needs or competence from protected or sensitive attributes. Exclude names and unnecessary demographic details from the final training data. Sample 2,000 personas spread across departments and occupations by running scripts/sample_personas.py into data/seeds.jsonl.  2.2: Generate raw data using personas The sdg/data_designer step then turns each persona into a customer message (scenario) and an agent reply (answer). You will need an LLM for the generator to use. If you are using a DGX Station, serve the model locally; if you are using a DGX Spark, it is recommended to use an NVIDIA-hosted API. For this example, we are serving Nemotron 3.5 Lightning locally with vLLM. docker run -d --gpus all --ipc=host --network host \ -v ~/.cache/huggingface:/root/.cache/huggingface vllm/vllm-openai:v0.26.0 \ --model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 \ --served-model-name lightning --port 8000 --reasoning-parser nemotron_v3 Point config/sdg_data_designer.yaml at that server: output_path: use-case-examples/french-public-service-assistant/data/sdg/sft_raw.jsonl num_records: 2000 seed_dataset: path: use-case-examples/french-public-service-assistant/data/seeds.jsonl providers: - name: local_vllm endpoint: http://localhost:8000/v1 #DGX Spark: https://integrate.api.nvidia.com/v1 provider_type: openai #DGX Spark: nvidia api_key: EMPTY #DGX Spark: NVIDIA_API_KEY extra_body: chat_template_kwargs: enable_thinking: false models: - alias: generator model: lightning #DGX Spark: nvidia/nemotron-3.5-lightning-30b-a3b provider: local_vllm inference_parameters: temperature: 0.8 max_tokens: 2048 #DGX Spark: 1024 max_parallel_requests: 64 #DGX Spark: 5 The most important setting is enable_thinking: false. Nemotron 3.5 Lightning produces reasoning traces by default, and this setting must sit under providers → extra_body → chat_template_kwargs for both options. If you put it under inference_parameters, the run fails. If you leave it out, reasoning text ends up in your training data. Each generated answer is also checked by a separate judge model. By default it calls the NVIDIA-hosted API, so set NVIDIA_API_KEY first; other judge endpoints are covered in the example README (Step 4 below). Preview eight records to start: uv run nemotron steps run sdg/data_designer \ -c use-case-examples/french-public-service-assistant/config/sdg_data_designer.yaml \ preview=true num_records=8 Check the records in data/sdg/sft_raw.jsonl against your behavior contract. When all eight pass, run the full generation. If running on a DGX Spark or any hosted API, start with 500 to 1,000 records.  2.3: Filter output Once finished, be sure to audit the output using scripts/audit_and_split.py. This filters out rows with any formatting issues, leftover reasoning, or potentially leaked persona names and then splits the rest 90/10 by source_uuid so the same persona never appears in both sets. The script saves the results to data/train.jsonl & data/validation.jsonl. Step 3: Fine-tune with LoRA Start from the NVIDIA NeMo Automodel based validated LoRA recipe for Lightning. We recommend starting out with changing the data paths and the training schedule, as the model and LoRA settings are tuned for this architecture. For the DGX Spark, reduce memory use by setting local_batch_size: 1, seq_length:8192, and --shm-size=16g.  From here, run training in the AutoModel container: docker run -it --rm --gpus all --ipc=host --shm-size=16g \ -v ~/Nemotron:/workspace/Nemotron -w /workspace/Nemotron \ nvcr.io/nvidia/nemo-automodel:26.08 \ automodel use-case-examples/french-public-service-assistant/config/peft_automodel.yaml --nproc-per-node 1 Validation loss should fall before it flattens. If it’s flat or rising from the start, check your data before continuing. Step 4: Merge and serve the model Keep the LoRA adapter for future iterations. Merge the adapter into the base checkpoint with the convert/merge_lora step. The exact command is in Step 7 of the example README and writes the merged model to checkpoints/lightning_fr_merged, which the command below serves. docker run -d --gpus all --ipc=host --network host \ -v ~/Nemotron:/workspace/Nemotron vllm/vllm-openai:v0.26.0 \ --model /workspace/Nemotron/use-case-examples/french-public-service-assistant/checkpoints/lightning_fr_merged \ --served-model-name lightning-fr --port 8001 The adapted model should now be available locally as lightning-fr and the base model should be available as lightning. On DGX Station, run both with -- --gpu-memory-utilization 0.45 so they fit side by side. On DGX Spark, serve only the adapted model and compare it against the hosted API or serve one model at a time (see Step 5.1). Step 5: Evaluate against the base model 5.1: Running the evaluation A lower loss isn’t enough to prove the post-trained model is better at the job. Send the same frozen test prompts to the base model and to the adapted model, then compare the answers one by one. This can be done through the use of an agent or a human—though it is recommended to evaluate with a human. Some example test cases can be found in eval/test_prompts.jsonl. Once you have these, serve both the base and the fine tuned models and run the evaluation from the repo root using: uv run python use-case-examples/french-public-service-assistant/scripts/compare_models.py \ --base-url http://localhost:8000/v1 --base-model lightning \ --tuned-url http://localhost:8001/v1 --tuned-model lightning-fr This writes eval/comparison.md, with the base and fine-tuned models’ answers side by side for each prompt. On a DGX Spark, serve one model at a time and run the script once per model, passing only --base-url/--base-model the first time and only --tuned-url/--tuned-model the second; the second run writes eval/comparison.md. 5.2: Reading the result Using the table above as a reference, your results should be along the lines of: Read every answer for hallucinations such as invented fees, deadlines, documents, and for replies that might break character such as “I am a language model.” Check each fact against the official pages. Get started In this tutorial, you walked through a complete post-training workflow for adapting a Nemotron open model to French-language public-service tasks. We walked through everything from defining a behavior contract and generating synthetic persona-grounded conversations, to fine-tuning with LoRA, merging the adapter, and evaluating the result against the base model. The same loop applies to any domain or language: Swap in different personas, a different behavior contract, and domain-specific reviewers. To go further—fine-tuning underlying model behavior with reinforcement learning or scaling training across multiple nodes—explore recipes based on NVIDIA NeMo RL and NVIDIA NeMo Gym open libraries. Explore open Nemotron models and datasets on Hugging Face. Use this calculator to estimate GPU needs, training time, and costs for customizing Nemotron models with SFT, LoRA, or GRPO. For voice-enabled applications, developers can complement the adapted LLM with NVIDIA Nemotron 3.5 ASR, a 600-million-parameter, streaming speech recognition model that transcribes audio as it arrives, including spoken French. Its coverage spans 40 language locales, with 32 available for transcription out of the box and eight requiring fine-tuning. Developers can also adapt recognition to specialized domains: Explore fine-tuning Nemotron ASR with NVIDIA Nemotron Speech agent skills for a companion workflow covering data preparation, training, and evaluation. Explore various usage cookbooks and use-case examples to customize Nemotron open models to power your domain-specific workflows. Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn, X, Discord, and YouTube. Visit the Nemotron developer page for resources to get started. Explore open Nemotron models and datasets on Hugging Face and Blueprints on build.nvidia.com. Engage with Nemotron livestreams, tutorials, and the developer community on the NVIDIA forum and the Nemotron channel on Discord.