Cloudflare Open-Sources Clef: A Multimodal Decision Model That Can Hear, See, and Decide

Which team should receive a customer support ticket? Is a machine making an unusual noise? These are recurring micro-decisions in agent workflows. Cloudflare's open-source decision model, Clef, converts input materials into probabilities across predefined options, letting software determine the next step.

Cloudflare introduced three major updates to the Clef family:

1. Released the multimodal decision model Clef-omni: Built by adapting Alibaba's open-source Qwen3-Omni-30B-A3B-Instruct mixture-of-experts (MoE) model, natively supporting audio (WAV/MP3) and video (MP4/WebM) inputs alongside text and images;
2. Clef-flash price slashed: The Workers AI hosted input price dropped to $0.038 / M tokens (a 57.8% reduction), while the hosted context window was reduced to 24,576 tokens (open-source weights were trained with support for 256k);
3. Hosted Clef significantly accelerated: With model weights unchanged, serving-layer optimizations reduced median latency across three benchmark tiers by approximately 40%–51%.

Understand inputs first, then score options directly

Take a repair ticket as an example: the system simultaneously asks "Is it urgent?", "Which team should handle it?", and "How severe is the impact?", listing allowed answers for each question.

Generative LLMs typically understand materials first and then write out answers token by token; Clef skips that final step. It passes the input through an understanding backbone network and forwards it to a compact scoring module: each candidate option first extracts relevant evidence from the input, then synthesizes information across different questions to score all options. This is two-stage attention routing. Scores for each question are then converted into probabilities, enabling business software to route tickets or escalate to humans.

During training, Cloudflare freezes the backbone and trains LoRA adapter parameters alongside the scoring module, optimizing option decisions while applying Brier loss to improve probability calibration. However, Softmax normalization into probabilities does not inherently guarantee that "an assessment reporting 0.8 has an 80% accuracy rate"; even with valid formatting, predictions can still be wrong.

It also reduces the effort of training separate models for every classification task. Routing tickets to billing, tech support, or sales today, and classifying forum posts into help requests, feedback, or ads tomorrow, can be approached by updating question and option descriptions in the request and testing the results, without first training a dedicated classifier for each new label set.

How Omni inspects visuals and listens to audio together

In equipment inspection, for instance, speech transcription captures "what people said," but motor grinding noise may not translate into text; visuals reveal component condition, while sound offers operational clues. Directly combining audio and visuals reduces the need for chained services and may preserve evidence lost in transcription. General multimodal LLMs that support corresponding media can also take inputs directly; Omni's distinction is applying this information to constrained option scoring.

A single API simplifies caller-side transcription, splitting, and concatenation, while the model internally encodes media and samples video frames:

• Video is sampled at 2 fps, which may miss brief flashes and other transient events;
• Video audio is parsed concurrently only when all videos in the same request have audio tracks;
• Inference uses only Qwen3-Omni's understanding backbone and audiovisual encoders. Speech generation weights remain in the open-source repository, but are retained without being loaded or used.

In Cloudflare's reported tests, pure text decisions had a median latency of approximately 130 ms, image inputs took about 150 ms, and audio clips took several hundred milliseconds; a 21-second video with audio was scored in approximately 1.5 s.

Accepting more input types does not make it stronger on every task

The three models serve distinct priorities: Flash (9B dense) focuses on low cost, Clef (27B dense) emphasizes accuracy and longer context, and Omni (30B-A3B MoE) adds audio and video inputs. Model selection depends on specific workloads rather than parameter counts alone.

In Cloudflare's internal evaluations, Omni achieved macro-F1 scores of 94.8% and 97.7% on BANKING77 and CLINC150+OOS intent classification, close to Clef; however, on smart home appliance instruction simulation, Flash reached 97.73%, noticeably outperforming Omni's 69.30%. On When2Call invocation timing, Jev led the group (80.97%, compared to Omni's 63.30%). These benchmarks are primarily text and instruction tests and cannot be equated with dedicated audio and video accuracy.

Metric definitions also matter: in invoice processing, Omni scored 82.0% for primary action accuracy, but only 60.2% for an exact match of the full action set. The former tracks key individual actions, while the latter requires the entire action set to match. If a workflow demands complete end-to-end correctness, an 82% figure cannot be taken as pipeline success.

How much cheaper did it get, and how much context was lost?

Current hosted configurations and pricing on Workers AI across the three models are as follows (all with no output token charges):

Flash's price cut came with a hosted window reduction from 64k to 24k. Cloudflare's rationale: only 0.24% of its production requests exceed 24k tokens. The open-source weights remain unchanged and supported a 256k window during training; workloads requiring longer context can use Clef or self-host Flash depending on available hardware.

Based on 1,000 input tokens per request across 1 million calls, Flash model costs dropped from $90 to $38. Note that pricing is per million tokens, not per million calls.

For multimodal inputs, Workers AI requires media submitted as Base64 data URLs and does not accept remote URLs:

• Media quotas: Up to 4 images, 4 audio clips (≤300 s per clip), and 2 videos (≤60 s per clip, sampled at 2 fps) per request; additional file size and pixel constraints are detailed in the Workers AI media specifications documentation;
• Token conversion: Audio uses approximately 13 tokens/s (780 tokens/minute); 480p video frames consume roughly 8,600 tokens per minute (reaching up to ~15,400 tokens/minute at maximum resolution); when audio track conditions are met, sound is converted separately as audio tokens;
• Limits and truncation rules: Converted media tokens and questions count together toward the context window, and requests are rejected immediately with an error if their sum exceeds the limit; only when media and questions fit within the limit while adding state pushes the total over budget will the text state undergo automatic truncation.

Assuming an evaluation of a 3-minute audio clip (~2,340 tokens) alongside 800 tokens of text questions (3,140 tokens total), at Clef-omni's $0.15 / M rate, calling it 1 million times incurs approximately $471 in pure inference model fees (excluding cloud infrastructure costs such as media storage and transfer bandwidth).

Why did Clef get faster without changing weights?

Clef's acceleration stems from the serving layer. Cloudflare reported median latencies across three input tiers:

With weights unchanged, migrating to SGLang was one of the optimizations (Cloudflare submitted PR #42721, slated for official support in SGLang 0.5.22).

Self-hosting deployment considerations

Weights are released under the Apache 2.0 license, but self-hosting still requires calculating hardware requirements. For Omni, 30B is total parameters and 3B is activated parameters per forward pass, which does not mean only 3B parameters need to be loaded. The model card specifies approximately 64 GB of VRAM for the bfloat16 backbone alone (tested on an H200), with additional memory headroom needed for inputs, KV cache, and runtime overhead. Clef already provides an SGLang launch recipe, whereas the Omni model card still lists it as "coming soon".

How to integrate into existing workflows

The interface uses state for input materials and questions to define queries and allowed answers. The three questions from the repair ticket illustrate the three supported question types:

• noul: Is it urgent? Returns the probability of "yes".
• choice: Which team should handle it? Returns a chosen option and probabilities across all options.
• score: How severe is the impact? Evaluates ordered grades starting from 0 and returns a probability-weighted expected score.

Clef is compatible with the Jev/SystemOne API and supports routing through AI Gateway. Existing compatible pipelines can switch models for drop-in evaluation; after benchmarking error rates, latency, and decision thresholds, teams can determine which branches are suited for automated execution.

Where is Cloudflare already using it?

Cloudflare's disclosed deployments of the Clef family include: identifying and closing spam issues in documentation repositories, auditing plugin libraries for phishing risks in EmDash, scanning data for PII such as government IDs in the DLP team, and detecting malicious domains in threat intelligence. These examples do not all specify Omni as the model used.

Decision models enable code to automate micro-judgments that resist exhaustive hardcoded rules: low-risk, verified-reliable branches execute automatically, while uncertain or high-risk cases escalate to human review. Tasks requiring open-ended writing and unconstrained planning remain the domain of generative models.