Skip to content

Configure an LLM Node

The LLM node is the heart of most chatbots in Open Chat Studio. It sends the conversation to an AI model and returns the model's reply.

This tutorial walks you through adding an LLM node to a pipeline and setting up the handful of options you'll use most often. By the end you will have a working LLM node that responds to messages using a model, a prompt, and conversation memory.

Before you start

You'll need:

Step 1: Open your chatbot

  1. Open your chatbot and view the Pipeline workflow. This is the visual canvas where you build your chatbot's conversation flow.
  2. New chatbots start with a single LLM node already on the pipeline, connected between the input and the output. If you see one, skip to Step 2.
Adding an LLM node manually

If your pipeline has no LLM node:

  1. On the pipeline, click the purple (plus) icon button on the top left.
  2. Choose LLM from the list of node types.

Step 2: Choose a model

The model determines which AI does the thinking.

  1. Click the LLM node edit button (or the Advanced link) to open its settings.
  2. Find the LLM Model dropdown and pick a model.
  3. If you're unsure which to choose, start with a general-purpose model and the defaults.

Not sure which model to pick?

See Choose an LLM Model for help deciding between general-purpose and reasoning models. Only models from a provider your team has configured will appear here.

Step 3: Write a prompt

The prompt tells the model who it is and how to respond. It's the single most important setting on the node.

  1. In the LLM node settings, find the prompt field.
  2. Describe the chatbot's role, tone, and any rules it should follow.

A good starting prompt is short and specific:

You are a friendly support assistant for Acme Co.
Answer questions about our products clearly and concisely.
If you don't know an answer, say so and suggest contacting support@acme.co.

Tip

Write instructions the way you'd brief a new teammate — state the goal, the tone, and what to do when unsure.

Step 4: Personalize with prompt variables (optional)

Prompt variables let you drop dynamic information into the prompt, so each participant gets a tailored response.

Insert a variable using curly braces. For example, to greet the participant by name:

You are a helpful assistant. The participant's details are: {participant_data.name}.
Use their name when appropriate and keep replies concise.

See Prompt variables for the full list of supported variables.

Step 5: Set the conversation history

An LLM node can be configured to remember earlier messages in the conversation so it can respond in context. This memory is controlled by the node's History setting, which defaults to Global so the model can see the full conversation — the right choice for a normal back-and-forth chatbot.

See Conversation History for how each history option behaves.

Step 6: Adjust the response style (optional)

Depending on the LLM model you chose, the node exposes a temperature or an effort setting that shapes how the model responds. The defaults work well for most chatbots, so you can safely skip this at first.

  • Temperature — lower for consistent, predictable answers; higher for more varied, creative ones.
  • Effort — how much a reasoning model "thinks" before answering.

For step-by-step guidance, see Adjust LLM Node Model Parameters.

Step 7: Save and test

  1. Click the x on the popup node settings to close the dialog and save your changes.
  2. Open the chat preview and send a test message.
  3. Confirm the reply reflects your prompt and model choice. Tweak the prompt and re-test until the responses look right.

Tip

When you're happy with the behaviour, create a version so participants keep a stable experience while you continue editing.

Next steps

You've built a working LLM node. When you're ready to do more, an LLM node can also: