# Why vLLM can't find a chat template for your model, and how to fix it > Source: https://netray.co/troubleshoot/vllm-chat-template-missing > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: On-Prem AI Troubleshooting ## Short answer The /v1/chat/completions endpoint needs a Jinja chat template to turn a messages array into the model's expected prompt format, and base pretrained checkpoints plus some older fine-tunes simply don't ship one. Fix it by supplying --chat-template pointing at a template file matching the model family, or by switching to the raw /v1/completions endpoint with a manually formatted prompt. ## Error Title vLLM error: no chat template found for this model ## Error Signature ValueError: As of transformers v4.44, default chat template is no longer allowed, so vLLM's chat api requires you to specify a chat template if the tokenizer does not define one ## Alternate Signatures - As of transformers v4.44, default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one. - jinja2.exceptions.TemplateError: chat_template is not defined - Cannot use chat template functions because tokenizer.chat_template is not set ## Category serving ## Stack - vllm - huggingface - transformers ## Affects vLLM 0.4 and later serving base or non-instruct checkpoints, or older fine-tunes saved before chat templates became mandatory metadata; the /v1/completions endpoint is unaffected ## Quick Fix ### Get chat completions working **Steps:** - Confirm the model is actually instruction-tuned; base or foundation checkpoints have no conversational format and shouldn't be called through /v1/chat/completions at all. - Check whether tokenizer_config.json in the model repo includes a chat_template field; if a sibling instruct checkpoint from the same family has one, that's your reference. - Download or write a Jinja chat template matching the model's expected special tokens and pass it with --chat-template /path/to/template.jinja. - Restart vLLM and re-test /v1/chat/completions with a simple messages array. - If you don't need chat formatting, use /v1/completions directly with a manually constructed prompt string instead. ## Symptoms - /v1/completions works fine but /v1/chat/completions errors immediately on the same model. - Error explicitly mentions chat_template, Jinja, or template not being defined. - The model is a base or foundation checkpoint, not one labeled Instruct or Chat. - A related instruct variant of the same model family serves chat completions correctly. ## Causes ### Model is a base pretrained checkpoint with no conversational fine-tuning or template **Likelihood:** most-common **Explanation:** Base checkpoints are trained purely for next-token prediction on raw text and were never taught a system, user, and assistant turn structure, so their tokenizer config has no chat_template field for vLLM's Jinja renderer to use when formatting the messages array into a single prompt string. **Fix:** Use /v1/completions with a manually built prompt instead of /v1/chat/completions, or switch to the model family's instruct or chat variant, which ships a template. ### Older fine-tuned checkpoint predates chat_template becoming a required tokenizer field **Likelihood:** common **Explanation:** Before this was standardized in Hugging Face tooling, many custom fine-tunes stored their prompt format only in a README or training script, not in tokenizer_config.json, so newer transformers or vLLM versions that require an explicit template find nothing to fall back to. **Fix:** Write a minimal Jinja chat template matching the exact format your fine-tune was trained on, with the correct system, user, and assistant delimiters, and pass it via --chat-template. **Commands:** - vllm serve MODEL_ID --chat-template ./my_template.jinja ### Chat template exists but isn't being picked up due to a path or loading issue **Likelihood:** common **Explanation:** If --chat-template points at a file vLLM can't find, such as a relative path issue inside a container, or the tokenizer is loaded from a local snapshot missing tokenizer_config.json, the template that does exist upstream on the Hugging Face repo never reaches the running server. **Fix:** Use an absolute path for --chat-template inside the container, and confirm tokenizer_config.json was actually downloaded alongside the weights. **Commands:** - ls -la /models/MODEL_ID/tokenizer_config.json ### Custom served-model-name or local snapshot detaches vLLM from the upstream repo's template **Likelihood:** occasional **Explanation:** When serving from a locally copied or converted checkpoint directory rather than pulling directly from the Hugging Face hub, any template metadata that lived only in the hub repo's tokenizer_config.json may not have been copied into the local snapshot. **Fix:** Explicitly copy tokenizer_config.json, or the full tokenizer files, from the source repo into your local model directory, or pass --chat-template explicitly. ### Template exists but references a tool or function-calling format the messages array doesn't provide **Likelihood:** rare **Explanation:** Some chat templates conditionally render tool-call blocks only when a tools field is present in the request. Malformed or partially specified requests can trigger template rendering errors that look like a missing-template problem but are actually a template logic branch failing. **Fix:** Test with a minimal messages-only request, without tools or functions, first to isolate whether the template itself is missing versus erroring on optional fields. ## Diagnostics ### Check whether the tokenizer config defines a chat template **Command:** python -c "from transformers import AutoTokenizer; t = AutoTokenizer.from_pretrained('MODEL_ID'); print(t.chat_template is not None)" **Interpretation:** False confirms the model genuinely has no built-in template; you must supply one or use /v1/completions. ### Test the raw completions endpoint as a fallback **Command:** curl http://localhost:8000/v1/completions -d "{\"model\":\"MODEL_ID\",\"prompt\":\"Hello\"}" **Interpretation:** If this works while chat completions fails, the issue is confined to chat templating, not the model or server generally. ### Inspect an instruct sibling model's template as a reference **Command:** python -c "from transformers import AutoTokenizer; t = AutoTokenizer.from_pretrained('INSTRUCT_MODEL_ID'); print(t.chat_template)" **Interpretation:** Comparing this against your base model's format tells you what special tokens and structure a compatible template needs. ## Prevention - Always verify a model has chat_template defined, or supply one, as part of your model-onboarding checklist, before it reaches production traffic. - Keep a small library of known-good chat templates per model family in version control alongside your deployment configs. - Prefer instruct or chat-tuned checkpoints for any workload that needs the chat completions API; reserve base checkpoints for completion-style or fine-tuning workloads. - Add an automated smoke test that calls /v1/chat/completions after every model swap, not just /v1/models. ## When To Escalate If you're regularly deploying custom or internally fine-tuned checkpoints without a consistent chat template process, standardize the fine-tuning pipeline to always emit tokenizer_config.json with a chat_template, so this stops being a recurring per-model fire drill. ## Faq ### What exactly is a chat template? **Answer:** It's a Jinja2 template stored in the tokenizer configuration that converts a structured messages array, with system, user, and assistant turns, into the single flat prompt string, with the model's specific special tokens, that the model was actually trained to expect. ### Can I just use any chat template as a workaround? **Answer:** No, using the wrong family's template will silently produce a badly formatted prompt with the wrong delimiter tokens, degrading output quality without necessarily erroring. Match the template to the exact model family, ideally its own tokenizer_config.json if a sibling instruct checkpoint exists. ### Does this affect the plain /v1/completions endpoint too? **Answer:** No, /v1/completions takes a raw prompt string directly from the caller and never needs a chat template. It's only /v1/chat/completions that requires one to convert the messages array format. ## Search terms vllm chat template error, chat_template not defined vllm, vllm no chat template found, vllm base model chat completions error, jinja chat template vllm --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json