← AI

A decision playbook for frontier model adaptation

Engineering note · Facts checked July 25, 2026 · 12 min read

Dated decision playbook · not a benchmark claim

The useful question is not “which model wins?” It is “which deployment contract lets this product improve?” I compare the current Kimi, Qwen, and GLM choices by availability, licensing, context, adaptation path, serving burden, and the evidence required before release. The result is a reproducible model-selection loop, not a leaderboard assembled from incomparable vendor scores.

Availability is part of the architecture. On July 25, Kimi K3 is available through an API, while Moonshot says downloadable weights will arrive July 27. I would not design a self-hosted K3 system as though those weights are already present. This page freezes that distinction so a later announcement cannot silently rewrite an earlier engineering decision.

The July 25 model snapshot

Parameter counts, active experts, and context windows describe resource envelopes; they do not prove task quality. Each candidate still has to run against the same private evaluation set, prompt, sampling policy, and output validator. With that caveat, the deployment surface is materially different across these families. As of this snapshot, Qwen3.7 Max and Plus are the newer hosted, proprietary Qwen tier; Qwen3.6-27B and Qwen3.6-35B-A3B are the latest Qwen weights available to download.

Candidate Status on Jul 25, 2026 Published shape What it enables now
Kimi K3 API available; weights promised for Jul 27, not yet downloadable 2.8T total; 16 of 896 experts active; native vision; 1M context Fast hosted evaluation of long, multimodal, agentic workflows
Qwen3.7 Max / Plus Hosted, proprietary models 1M-context hosted product tier Managed deployment when owning weights is not a requirement
Qwen3.6-27B Downloadable; Apache-2.0 Dense, multimodal; 262K native context A simpler open-weight target for adapters, evals, and controlled serving
Qwen3.6-35B-A3B Downloadable; Apache-2.0 35B total / 3B active MoE; multimodal; 262K native context MoE experimentation when active compute and total weight residency are modeled separately
GLM-5.2 Downloadable; MIT 744B total / 40B active; 1M context Open-weight frontier work when a fleet-scale serving plan is justified

Moonshot says Kimi K3 deployment uses 64 or more accelerators. That is a vendor statement, not a hardware recommendation. Likewise, “open weights” says nothing about whether a model is affordable at the required latency, concurrency, or availability target.

Select the deployment contract before the model

I start with constraints that a benchmark cannot answer: Must the weights stay inside a controlled environment? Is an API acceptable? Is vision part of the request path? Does the application genuinely use hundreds of thousands of tokens, or would retrieval produce better evidence at lower cost? The candidate list comes after those decisions.

Decision path for choosing a hosted or downloadable model A flow chart starts with the product constraint. Private weights lead to Qwen3.6 or GLM-5.2 and a hardware-fit test. An acceptable hosted API leads to Kimi K3 or Qwen3.7. Both routes converge on task evaluation, cost, and operational gates before release. What must the product control? data boundary · weights · latency · modality own weights required hosted API acceptable Downloadable path Qwen3.6-27B / 35B-A3B or GLM-5.2 when fleet cost fits Hosted path Kimi K3 API / Qwen3.7 Max or Plus measure privacy, rate, and lock-in constraints Run one frozen task suite same inputs · validator · prompt budget · sampling Release only if quality, retention, safety, latency, and cost gates pass otherwise change the prompt, data, adaptation method, or candidate

A model enters the evaluation only after its deployment contract is feasible; impressive scores cannot repair an impossible data or hardware boundary.

My default shortlist. Prototype the product against hosted APIs when that is allowed; use Qwen3.6 for an open-weight adaptation baseline; consider GLM-5.2 only when its measured quality offsets the operational burden. Re-run the shortlist when Kimi K3 weights actually ship rather than pre-committing to an announcement.

Treat the dataset as a versioned product

Fine-tuning is downstream of data governance. Every record needs a source, license or consent basis, transformation history, and split assignment. Deduplication has to happen before the train/evaluation boundary is frozen; otherwise near-duplicates turn an evaluation into a memory test. Personally identifiable and secret material is removed before it reaches either a hosted teacher or a training job.

Auditable data lifecycle for model adaptation Source records pass through rights checks, normalization, split-safe deduplication, synthetic expansion, verification, and versioning. A frozen evaluation set reports failure clusters back to the next data version without entering training. Source origin · consent license · timestamp Normalize schema · PII language · quality Dedupe exact + semantic before split freeze Expand target failures record generator Verify + version tests · filters · review manifest · hashes · lineage Train / adapt candidate data version + code SHA + base revision Frozen evaluation never used as generation context failure clusters inform the next version, not its labels

The evaluation set can reveal where the system fails, but it never becomes teacher context or training data. That boundary is what preserves a meaningful comparison across versions.

Synthetic data without dataset laundering

Synthetic examples are useful when they cover a named gap: rare tool failures, long-tail visual conditions, schema edge cases, or code tasks with executable checks. For every generated record I retain the teacher model and revision, prompt template, decoding settings, seed, source record IDs, and verifier results. I prefer deterministic checks (compilation, unit tests, simulators, database constraints, or exact structured-output validation) over a second model merely saying the first model looks correct.

Generation is sampled from training-side failure clusters, never from a hidden answer key. Filters reject duplicates, unverifiable claims, style artifacts, and implausibly easy examples. The final mixture retains high-quality human or source-grounded anchor data, and an ablation compares source-only, synthetic-only, and blended variants. More tokens are not evidence; a measurable gain on a contamination-resistant holdout is.

Choose the smallest adaptation that can pass

Prompting, tool use, and retrieval come first because they are reversible and preserve a clean base-model comparison. Fine-tuning earns its cost when the failure is behavioral or representational: a stable output contract, domain language, visual conventions, tool selection, or a policy that repeated prompting does not repair.

Method Use it when Primary risk Required proof
Prompt + retrieval Facts change often, citations matter, or the desired behavior already exists Context cost and retrieval misses Retrieval recall, groundedness, latency, and abstention
LoRA A focused behavior or domain shift can fit in low-rank updates Adapter overfit and merge/serving mismatch Adapter-versus-base ablation plus retention suite
QLoRA Iteration is memory constrained and a quantized frozen base is acceptable Training throughput and precision-sensitive regressions Compare with LoRA on a smaller controlled run; re-evaluate the served artifact
Full fine-tune The distribution shift is broad, the dataset is large and clean, and adapters plateau Forgetting, optimizer memory, recovery cost Matched-budget adapter baseline, broad retention gates, restartable checkpoints
Preference / RL stage A verifiable reward captures an objective SFT cannot express Reward hacking and capability collapse Held-out reward, task success, KL/retention, and adversarial failure review

Tooling by job, not by logo

  • LLaMA-Factory is a strong recipe layer for continued pretraining, SFT, preference optimization, RL, full tuning, LoRA, and QLoRA. Its companion ecosystem also points to dataset construction tools, but generated data still needs the provenance loop above.
  • Axolotl is useful for configuration-driven, distributed full and parameter-efficient runs, including preference and reinforcement methods. Model-family support is version-sensitive: pin the revision, inspect the exact architecture mapping, and run a forward/backward smoke test before renting a cluster.
  • ms-swift is the first path I would test for GLM-5.2 SFT and policy optimization; slime is a GLM-oriented reinforcement-learning path. I would preserve the versions named by Z.ai (ms-swift 4.4 or later and slime 0.3 or later) in the run manifest.
  • vLLM and SGLang are serving engines, not training frameworks. Export correctness, chat templates, vision preprocessing, quantization, tensor/expert parallelism, and tool calling must be tested in the engine that receives production traffic. For GLM-5.2, Z.ai's published compatibility floor is vLLM 0.23 or later, or SGLang 0.5.13.post1 or later; the actual deployment should still pin and record an exact version.

Optimize cost per accepted task

Price per million tokens is only one line in the ledger. I compare candidates by cost per accepted task: total API or fleet cost divided by outputs that pass the task validator. A cheap model that retries three times, occupies a human reviewer, or emits invalid tool calls can be more expensive than a higher-priced model that finishes once.

accepted-task cost = (data + training + serving + retries + review + eval operations) / accepted outputs

The training ledger includes data acquisition and annotation, filtering, failed runs, checkpoint storage, evaluation inference, and engineer time, not only successful GPU-hours. The serving ledger includes weight residency, KV-cache capacity, prefill and decode latency, idle headroom, observability, and failover. For an MoE, I track active compute separately from the memory and network cost of hosting all experts. Long context is measured with the actual prompt distribution because advertised capacity is not a latency SLO.

The break-even experiment is simple: route a fixed, representative workload through the hosted baseline and the self-hosted candidate, then compare accepted-task cost at expected average and peak load. Include a low-utilization week. If the self-hosted system wins only when every accelerator is perfectly occupied, it does not yet win.

Make release a sequence of gates

A single aggregate score hides the regression that matters. I keep each gate separate, report uncertainty where sampling is involved, and attach failures to concrete examples. A candidate can improve domain performance and still be rejected for retention, safety, invalid structured output, or an unacceptable latency tail.

Five model release gates with rollback A candidate artifact proceeds through task quality, retention, safety, serving, and cost gates. Any failed gate sends the artifact back to the experiment ledger; passing all five creates a canary release that remains monitored and reversible. 01 · Task success · validity calibration 02 · Retain general tasks anchor behavior 03 · Safety abuse · privacy prompt attacks 04 · Serve TTFT · throughput tail · recovery 05 · Cost accepted task average + peak Canary, monitor, rollback versioned artifact + routing policy Experiment ledger failure example · hypothesis · next run any failed gate

The production artifact includes the base revision, adapters or merged weights, tokenizer, chat template, preprocessing, engine version, quantization recipe, and routing configuration.

The minimum evaluation packet

  • Task quality: a frozen golden set, real traffic replay with sensitive fields removed, and a deterministic validator wherever possible.
  • Retention: capabilities the base already had and the product still depends on, reported next to the domain gain rather than averaged away.
  • Safety and reliability: tool permissions, prompt injection, data leakage, refusal boundaries, malformed output, timeouts, and fallbacks.
  • Serving: time to first token, inter-token latency, throughput, memory, queueing, cold start, and p50/p95/p99 under the expected request mix.
  • Economics: accepted-task cost at average and peak load, with retries and human escalation included.

A portfolio-quality implementation

I would turn this playbook into one reproducible repository rather than four unrelated demos. It would contain a normalized task suite, provider and local-engine adapters, immutable dataset manifests, one small Qwen3.6 adapter run, the untouched base baseline, an export-and-serve smoke test, and a generated report that binds every metric to a git commit and artifact digest.

The employer-facing result is not “I called the newest API.” It is a decision record showing why a candidate entered the test, how the data could legally and technically be reproduced, which adaptation beat the baseline, what regressed, what one accepted result cost, and how the production route rolls back. That is the work required to make frontier models behave like an engineered subsystem.

Primary sources

  1. Moonshot AI, Kimi K3 quickstart : API surface and current access path.
  2. Moonshot AI, Kimi K3 announcement : architecture, modalities, context, deployment statement, and announced weight-release date.
  3. Qwen, Qwen3.7 announcement : hosted Max and Plus product family.
  4. QwenLM, Qwen3.6 repository : downloadable models, architecture, context, multimodal support, and license.
  5. Qwen3.6-27B model card and Qwen3.6-35B-A3B model card : exact downloadable artifacts and configuration details.
  6. Z.ai, GLM-5.2 announcement : published model shape, context, training, and serving paths.
  7. Z.ai, GLM-5 repository and GLM-5.2 model card : weights, license, and inference guidance.
  8. LLaMA-Factory, Axolotl documentation, ms-swift, and slime : adaptation and reinforcement-learning toolchains.
  9. vLLM documentation and SGLang documentation : serving engines and deployment features.