Conversation History¶
Definition
An LLM node's conversation memory is controlled by two independent, per-node settings. History decides which messages the model sees. History Mode decides how those messages are trimmed once there are too many. They apply only to nodes with an LLM response.
AI models have no memory of their own — each request OCS sends is self-contained. Open Chat Studio always stores the full conversation in the chatbot's session, however a node is configured. History and History Mode only control what's sent to the model, not what's saved.
History¶
The History setting controls which messages a node's LLM sees: its own, the whole conversation, or a shared slice of it.
History Options¶
| History | What the LLM sees |
|---|---|
| Global | The full conversation the participant sees. This is the default for an LLM Node. |
| Node | Only this node's own past inputs and outputs. This is the default for an LLM Router Node. |
| Named | A shared history that specific nodes contribute to together. |
| No History | Nothing from earlier in the conversation. |
Each option suits a different situation. Complex pipelines usually use different History settings across nodes.
Global¶
Nodes with Global history will supply the conversational history that the participant would see to the LLM. The simple example uses a global history as the participant is interacting directly with a single LLM.
Node¶
Node history will maintain a specific history for this particular node. The input to the node will be saved, along with the output from the LLM.
LLM output is not necessarily the same as node output
In a LLM Router node, the output from the node will be the same as the input to that node. That is, once it has done its routing, it will be a passthrough for the input. The output of the LLM however, will be the classification label. This is an important distinction to keep in mind.
A common use case is in a LLM Router node. There, you may want a history of the node outputs, for example to track which part of the chatbot the participant is using. The history should then use LLM outputs. This stops the LLM being shown the wrong type of output as examples in its prompt (known as few-shot examples).
Named¶
This option allows you to specify a specific, named history that can be shared between nodes. Each node using the same shared history contributes its input and LLM output to it.
Named history is updated immediately
If there are multiple nodes in series that use the same Named history, then each node will add to the history. This will result in multiple new history entries for every processed participant message.
The most common use case for this will be when you have multiple parallel nodes after an LLM Router. In the Workflow Cookbook, the general, quiz, and roleplay LLM nodes would likely share one history. This gives each node visibility into the larger conversation.
Note that for this particular example, each of the nodes could use a Global history to achieve the same thing. However, a translation or formatting node might come before the final output.
Then the Named history option lets the interim nodes share a history in the original language or formatting.
No History¶
Choosing No History means that when a completion is requested from the LLM, no conversational history will be supplied. One common use case might be a formatting or translation node where the previous history may not be applicable to generating the correct output.
History Mode¶
How much history a node sends matters because the prompt, the conversation history, and the reply share the same token budget. Sending more history gives the model more context, but costs more tokens and can slow responses down.
Once a node has more history than fits in the model's token budget, OCS must discard or condense older messages. History Mode controls what happens to older messages when that limit is reached. It only has an effect when History is set to Node, Global, or Named — with No History there's nothing to trim.
History Mode Options¶
| History Mode | What happens to older messages |
|---|---|
| Summarize | Condensed into a summary once the token limit is reached. This is the default. |
| Truncate Tokens | Dropped once the token limit is reached, until back under it. |
| Max History Length | Dropped once the message count exceeds N. |
Summarize¶
Compresses older messages into a summary once the token limit is reached, keeping the most recent messages intact.
Token Limit: Maximum number of tokens allowed before summarization occurs.
Truncate Tokens¶
Removes older messages once the token limit is reached, until the total is back under the threshold.
Token Limit: Maximum number of tokens allowed before truncation occurs.
Max History Length¶
Keeps only the most recent N messages, dropping older ones once that limit is reached.
Max History Length: Number of recent messages to keep. Only this number of messages will be sent to the LLM.