Connect Simon Willison's llm CLI to Gonka Broker

llm is Simon Willison’s command-line tool and Python library for running prompts against large language models. It reaches Gonka Broker through its OpenAI-compatible interface, so prompts, piping, chat and the Python API all work unchanged; you only choose how to point llm at Gonka Broker.

Prerequisites

• A Gonka API key (starts with gnk-prx-). See Create a Gonka API Key.
• Python 3.10+ with llm installed (pip install llm or brew install llm).

Use the dedicated plugin (recommended)

The llm-gonkabroker plugin registers Gonka Broker‘s models in llm for you, with no base URL to configure. Models are discovered automatically from Gonka Broker‘s public model list.

Install it into the same environment as llm:

llm install llm-gonkabroker

Set your key (paste your gnk-prx-… key when prompted):

llm keys set gonkabroker

Run a prompt; model ids are prefixed with gonkabroker/:

llm -m 'gonkabroker/MiniMaxAI/MiniMax-M2.7' 'Hello!'

List the available models, pipe content in, or start a chat:

llm models list | grep gonkabroker
cat article.md | llm -m 'gonkabroker/MiniMaxAI/MiniMax-M2.7' -s 'Summarize this'
llm chat -m 'gonkabroker/MiniMaxAI/MiniMax-M2.7'

The key can also be supplied via the GONKABROKER_API_KEY environment variable instead of llm keys set. Models only appear in llm models list once a key is configured.

Use the OpenAI-compatible config

This works with any setup, no plugin required. llm can talk to any OpenAI-compatible endpoint through its extra-openai-models.yaml file.

Find the config directory and edit (or create) extra-openai-models.yaml:

dirname "$(llm logs path)"   # prints the llm config directory

extra-openai-models.yaml

- model_id: Gonka-minimax
  model_name: MiniMaxAI/MiniMax-M2.7
  api_base: https://proxy.gonkabroker.com/v1
  api_key_name: Gonka

Store your key under that name, then run a prompt:

llm keys set Gonka
llm -m Gonka-minimax 'Hello!'

model_name must match a Gonka-supported model id (see Supported Models). Add one entry per model you want to use.

Piping, chat and Python

All of llm’s features work the same regardless of which option above you use: pipe stdin with -s for a system prompt, start an interactive session with llm chat, and call it from Python with llm.get_model(...). The OpenAI-compatible transport handles streaming under the hood.

Verify

Run any prompt above with your key configured. Printed output confirms the connection. llm logs -n 1 shows the last response and which model served it.

Troubleshooting

• 401 / invalid API key: wrong or paused key, or the key was stored under a different name than the model references.
• Model not found: the model id (or model_name) must match a supported model.