---
myst:
html_meta:
title: AutoRAG - OpenAI LLM
description: Use OpenAI LLM in AutoRAG. This is optimized to use OpenAI model in AutoRAG.
keywords: AutoRAG,RAG,LLM,generator,OpenAI,GPT,gpt-4,gpt-3.5
---
# OpenAI LLM
The `openai_llm` module is optimized openai llm module for AutoRAG.
## Why use `openai_llm` module?
There are several advantages using `openai` module in AutoRAG.
### 1. Auto-truncate prompt
Sometimes, prompt might exceed a token limitation of the model.
It will occur server-side error, and all your answer results will be gone.
To prevent this, `openai_llm` module truncate prompt to the max length of gpt model.
### 2. Accurate token output
In `llama_index_llm` module, it does not return proper tokens. It just return pseudo token using GPT2 tokenizer.
When you use `openai_llm` module, you can get real tokens that used in gpt model.
In the future, there will be a module that uses token for boosting RAG performance.
### 3. Accurate log prob output
In `llama_index_llm` module, it does not return proper log probs since llama index does not support it.
With `openai_llm` module, you can get real log probability to every token of generated answers.
In the future, there will be some modules that use log probability, like answer filter.
## Support chat prompt
From v0.3.19, you can use chat prompt with `openai_llm` module.
For using chat prompt, you have to use `chat_fstring` module for prompt maker.
## Support GPT-5 models
From v0.3.22, you can use GPT-5 models with `openai_llm` module.
You can set reasoning efforts and verbosity level.
The latest GPT-5 family models are supported, including the gpt-5.6 family
(`gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, and the `gpt-5.6` alias),
`gpt-5.5`, and `gpt-5.4`, all with a 1.05M token context window.
Dated snapshots (e.g. `gpt-5.6-sol-2026-07-09`) resolve to their base model's
context window automatically.
```yaml
modules:
- module_type: openai_llm
llm: [ gpt-5.6-sol ]
max_tokens: 512
reasoning:
effort: high
verbosity: low
```
GPT-5.6 models support reasoning efforts from `none` up to `max`
(`none`, `low`, `medium`, `high`, `xhigh`, `max`).
## Retired models
Retired OpenAI models are intentionally not supported and raise a clear error
at module initialization. This follows the official
[OpenAI deprecations page](https://developers.openai.com/api/docs/deprecations).
Notable removals: `gpt-5`, `gpt-5.1`, `gpt-5-mini`, `gpt-5-nano`,
`gpt-5-chat-latest`, `chatgpt-4o-latest`, `o1-preview`, `o1-mini`, `gpt-4-32k`,
the gpt-4 vision/turbo preview snapshots, and the `gpt-3.5-turbo-0613`/`16k`
snapshots. Migrate to the gpt-5.4/5.5/5.6 family (e.g. `gpt-5.6-terra` as a
drop-in for `gpt-5`/`gpt-5.1`, `gpt-5.4-mini` for `gpt-5-mini`).
## **Module Parameters**
- **llm**: You can type your 'model name' at here. For example, `gpt-4-turbo-2024-04-09` or `gpt-3.5-turbo-16k`
- **batch**: The batch size of openai api call. You should decrease when you got token limit error.
- **truncate**: Whether you truncate input prompts to model's max length. Default is True. Recommend you to keep this
True.
- **api_key**: OpenAI API key. You can also set this to env variable `OPENAI_API_KEY`.
- And all parameters
from [OpenAI Chat Completion](https://platform.openai.com/docs/api-reference/chat/create)
without `n`, `logprobs`, `stream` and `top_logprobs`.
## **Example config.yaml**
```yaml
modules:
- module_type: openai_llm
llm: [ gpt-3.5-turbo, gpt-4-turbo-2024-04-09 ]
temperature: [ 0.1, 1.0 ]
max_tokens: 512
```