How to Run Open-Source LLMs with Together AI

Together AI is a cloud platform that serves open-source models — Llama, Qwen, DeepSeek, Mixtral and more — through an API that mirrors OpenAI’s. You get a key, point an OpenAI client at their URL, and call models you could not easily host yourself. New accounts get a starting credit, and pricing is per token with some models at a few cents per million tokens.

The appeal is cost and choice. If you want a capable open model without managing GPUs, Together AI is one of the simpler routes. It also offers fine-tuning and dedicated endpoints for heavier use.

What is Together AI?

Together AI is an inference and training platform for open models. It exposes a chat completions API compatible with OpenAI’s SDK, plus endpoints for embeddings and images. Models run on their infrastructure, so you send a request and get tokens back without provisioning hardware.

How do you get started?

1. Create an account and generate an API key from the dashboard.
2. List available models via the models endpoint and note the exact IDs.
3. Call the chat completions endpoint with a model ID like meta-llama/Llama-3.3-70B-Instruct-Turbo.
4. Use the OpenAI Python client by setting base_url to the Together endpoint.

Because the interface matches OpenAI, moving existing code over is mostly a one-line change.

Which models are available?

The catalog covers large open models across sizes and licenses, from small fast ones to 70B-class and larger mixture-of-experts models. Model IDs follow a vendor/name pattern. Availability changes as new releases land, so always pull the live model list rather than hardcoding an ID.

How does pricing work?

You pay per input and output token, with rates that vary by model — small models are cheap, large ones cost more. Some models offer a free or heavily discounted tier for experimentation. The starting credit lets you benchmark before committing. For steady high volume, dedicated endpoints give fixed capacity but a higher floor price.

What are the limitations?

You get less control than running the model yourself — no custom quantization or kernel tweaks. Some models have usage restrictions tied to their licenses. And though the API looks like OpenAI’s, edge features (certain tool-calling or response formats) may differ per model, so test your exact use case rather than assuming parity.

Key Takeaways

• Together AI serves 100+ open-source models behind an OpenAI-compatible API.
• Swap in existing OpenAI SDK code by changing the base URL.
• Pricing is per token and varies by model; new accounts get starting credit.
• Also offers embeddings, images and fine-tuning.
• Less low-level control than self-hosting; check license terms per model.

Last updated October 2026. Details at Together AI.