Runtime as featured inForbesRead the article
Kimi K3.1 Pro specs
Moonshot AI logo

How to run Kimi K3.1 Pro

Bedrock, Vertex AI, or your own GPUs · Checked November 21, 2026

Kimi K3.1 Pro has 2.8T parameters, so it needs about 6,720GB of GPU memory at full precision and about 3,360GB at FP8. A practical starting point is 4-bit quantized on Multiple 8x H200 nodes.

You have three ways to run it: a managed host that already serves it, your own GPUs in your cloud, or both behind one endpoint. Then you point your agents at it. Here is what each path takes.

Kimi K3.1 Pro hardware requirements

PrecisionMemory neededSmallest setup that fits
BF16 (full precision)~6,720GBMultiple 8x H200 nodes (about 6)
FP8~3,360GBMultiple 8x H200 nodes (about 3)
4-bit quantized~1,680GBMultiple 8x H200 nodes (about 2)

Our estimates: 2.8T parameters times bytes per parameter, plus 20% for the KV cache and the serving runtime. As a mixture-of-experts model, Kimi K3.1 Pro activates only 104B per token, which makes it fast, but every expert still has to sit in memory. Using the full 1M-token context needs more. Check the model card for the precision the weights ship in and any recommended settings.

Where to run Kimi K3.1 Pro

01Amazon Bedrock

Available

Kimi K3.1 Pro is available on Amazon Bedrock as a managed, serverless model (listed as Kimi K3.1 Pro - Highspeed). You call it from your AWS account with IAM, and AWS runs the GPUs. Source

02Google Cloud Vertex AI

Not confirmed as of November 21, 2026

We could not confirm Kimi K3.1 Pro in Vertex AI Model Garden. To keep it in Google Cloud, serve it yourself on GPUs in your project (GKE or Compute Engine) with the setup below.

03Self-host on your own GPUs

vLLM and SGLang support confirmed

Download moonshotai/Kimi-K3.1-Pro from Hugging Face and serve it with an inference server that exposes an OpenAI-compatible endpoint. vLLM and SGLang support it. Start with 4-bit quantized on Multiple 8x H200 nodes (about 2), then load-test with your real prompts before you size for production.

  • Keep the endpoint private to your VPC, so prompts and payment data never leave your cloud.
  • Read the Kimi K3.1 Pro License before you deploy, since custom licenses can carry conditions.
  • Run two replicas or keep a hosted model as a fallback, so agents keep working during an upgrade.

Run Kimi K3.1 Pro with Runtime

Serving the model is half the job. The other half is giving agents a safe place to use it. Runtime is the AI agent harness for payment and fintech teams, and it is model-neutral: connect Kimi K3.1 Pro from Bedrock, Vertex AI, or your own endpoint, and your agents use it next to Claude, GPT, or Gemini.

  • In your cloud. Runtime runs in your AWS, GCP, or Azure account or fully self-hosted, next to the model.
  • Routed by evals. Evals show which steps Kimi K3.1 Pro handles as well as a frontier model, so routine work moves to the cheaper model.
  • Fallbacks. If your endpoint is down or slow, agents fall back to another model instead of stopping.
  • Guardrails. Agents start read-only, wait for approval before money moves, and every run is recorded.

Harnesses that can drive Kimi K3.1 Pro on Runtime:

Claude CodeKimi CodeOpenCode

Running Kimi K3.1 Pro: FAQ

How much GPU memory does Kimi K3.1 Pro need?

+

About 6720GB at BF16, 3360GB at FP8, and 1680GB at 4 bits. These are our estimates: the weights plus 20% headroom. Long contexts need more memory for the KV cache.

What is the cheapest way to self-host Kimi K3.1 Pro?

+

Start with 4-bit quantized on Multiple 8x H200 nodes (about 2). Quantizing further saves memory but can cost accuracy, so test it on your own tasks first.

Is Kimi K3.1 Pro available on Amazon Bedrock?

+

Yes. Kimi K3.1 Pro is available on Amazon Bedrock as a managed, serverless model (listed as Kimi K3.1 Pro - Highspeed).

Is Kimi K3.1 Pro available on Google Vertex AI?

+

As of November 21, 2026, we could not confirm Kimi K3.1 Pro in Vertex AI Model Garden. You can still run it in your Google Cloud project on your own GPUs (GKE or Compute Engine) with vLLM and SGLang.

Can I use Kimi K3.1 Pro with Runtime?

+

Yes. Runtime is model-neutral. Serve Kimi K3.1 Pro from Bedrock, Vertex AI, or your own endpoint, connect it to Runtime, and agents use it through a harness like Claude Code. Your data stays in your cloud, and evals tell you which steps it can take over from a more expensive model.