Ollama

FlowKit supports local LLMs via Ollama with the OllamaAdapter.

Setup

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Pull a model
ollama pull llama3.2
ollama pull qwen3:4b

Usage

import { OllamaAdapter } from "@andresaya/flowkit";

const adapter = new OllamaAdapter({
  model: "llama3.2",
  baseUrl: "http://localhost:11434", // Optional, this is default
});

const engine = new FlowEngine(flow, { llm: adapter, storage });

Configuration

interface OllamaConfig {
  /** Ollama base URL (default: http://localhost:11434) */
  baseUrl?: string;
  /** Model to use (default: llama3.2) */
  model?: string;
  /** Temperature (default: 0) */
  temperature?: number;
  /** Timeout in ms (default: 60000) */
  timeout?: number;
}

Recommended Models

Model	Size	Best For
qwen3:4b	4B	Best balance of speed and JSON extraction
qwen3:8b	8B	Better reasoning
llama3.2	3B	Fast, good for simple flows
llama3.1:8b	8B	Good general purpose
mistral	7B	Good multilingual support

Example

import { 
  agent, flow, FlowEngine, MemoryStorage, OllamaAdapter, 
  name, yesNo 
} from "@andresaya/flowkit";

const bot = agent("Alex")
  .personality("friendly")
  .build();

const myFlow = flow("greeting", bot)
  .ask("name", "What's your name?", name(), "user_name")
  .then("confirm")
  .ask("confirm", "Nice to meet you {{user_name}}! Need help?", yesNo(), "needs_help")
  .when({ yes: "help", no: "bye" })
  .say("help", "How can I help?")
  .done()
  .say("bye", "Goodbye!")
  .done()
  .build();

const engine = new FlowEngine(myFlow, {
  llm: new OllamaAdapter({ model: "llama3.2" }),
  storage: new MemoryStorage(),
});

const result = await engine.start("session-1");
console.log(result.message);

Features

Feature	Supported
Streaming	No
Tool Calling	Yes
JSON Mode	Yes
Local/Free	Yes

Tool Calling

FlowKit supports tool calling with Ollama. This requires a model that supports tools:

Supported models for tool calling:

• Llama 3.1+
• Qwen 3 / Qwen 2.5
• Mistral Nemo
• Command-R+
• Firefunction v2

const engine = new FlowEngine(myFlow, {
  llm: new OllamaAdapter({ 
    model: "llama3.1"  // Use a model that supports tools
  }),
  storage: new MemoryStorage(),
  tools: [{
    name: "get_weather",
    description: "Get the current weather for a city",
    execute: async (params) => {
      // Your implementation
      return { temperature: 22, condition: "sunny" };
    },
    parameters: {
      type: "object",
      properties: {
        city: { type: "string", description: "City name" }
      },
      required: ["city"]
    }
  }]
});

Note: Make sure to use ollama pull <model> to download a model that supports tools before using tool calling.

Tips

1. Use qwen3:4b for strict mode - Better JSON extraction than llama models
2. Keep temperature at 0 - For consistent, deterministic responses
3. Model must be running - Make sure Ollama is running: ollama serve
4. Pull models first - Download with ollama pull <model> before using

Troubleshooting

Connection Refused

Error: Ollama error 500: connection refused

Solution: Make sure Ollama is running: ollama serve

Model Not Found

Error: model 'xyz' not found

Solution: Pull the model first: ollama pull xyz