One model for text, images, video, and mixed inputs

All three versions accept the same inputs; they differ in model size, full vector dimensions, and runtime cost.

The 2B, 4B, and 9B models output full vectors of 2048, 2560, and 4096 dimensions, respectively.

Each version handles plain text, images, video, visual documents, and content where text and images alternate. Queries and candidate items can also carry task instructions, so the same image can get different vector representations depending on whether the goal is "find visually similar products" or "find a page that answers this question."

These vectors are well-suited for the first recall stage of search and recommendation systems: they can pull out a small, highly relevant candidate set from a massive pool, which a more complex ranking model then processes. They also work for classification, clustering, and Agent memory, where comparing content similarity is key.

Projecting diverse content into a shared coordinate space

Text, images, and video have fundamentally different internal representations. WeMM-Embedding processes all of them with a single native multimodal backbone, preserving the original order of each segment in the input, which allows it to directly understand interleaved formats like "text-image-text."

Eventually, the various inputs converge at a dedicated position.

The output is normalized and can be used directly for cosine similarity calculations.

During processing, the model inserts a special <embedding> token into the input sequence. It reads the hidden state at that position in the final layer and applies L2 normalization to produce the final vector. This token effectively asks the model to compress the preceding text, images, video frames, and task instructions into a single coordinate.

You can also include multiple embedding tokens in the same sequence to represent different segments or granularities. This way, long videos, visual documents, and mixed-content sequences don't have to be crudely split into unrelated modality chunks.

Two-stage training that makes "similar" mean "truly relevant"

Putting different content into the same vector space only solves the "can we compare it" problem.

The two-stage training process tackles "comparing accurately."

The first stage uses hundreds of millions of multimodal pairs to build broad coverage, bringing text, images, video, visual documents, and interleaved content into a common space. It uses contrastive learning to pull positive examples together and push negative examples apart. Each batch contains only one task type, which prevents cross-task samples from becoming easy, low-value negatives.

In real business data, content treated as a negative example might actually be relevant to the query. Repetition-aware masking filters out these false negatives, while graded relevance goes further to distinguish fully relevant, partially relevant, topically related, and unrelated cases, so similarity isn't just binary 0 or 1.

The second stage shifts focus from coverage to difficult decision boundaries. Data goes through semantic resampling, quality cleaning, and hard negative mining, and then incorporates reranker ranking signals. The 9B model distills the similarity distribution among samples into the 2B and 4B models, while 9B itself improves through model merging. The resulting ranking signal is more nuanced than coarse topic categories, making it better suited for retrieval and recommendation.

Top scores on both MMEB benchmarks for 9B; already live in WeChat

This training regime produces results in three areas: public benchmarks, internal WeChat tasks, and live business metrics.

The 9B model targets maximum quality, while the 2B and 4B versions focus on balancing performance with deployment cost.

On MMEB-v2, the 2B, 4B, and 9B models score 77.9, 79.2, and 80.6, respectively. For context, 2B-class comparison models in the table range from 47.8 to 74.8, while 8B-to-9B comparison models range from 53.2 to 78.4. Notably, Qwen3-VL-Embedding-8B scores 77.8, placing WeMM-Embedding-2B's 77.9 slightly ahead. The 9B model's subscores for image, video, and visual document are 81.9, 74.3, and 83.3, respectively.

MMEB-v2 aggregates results from 78 datasets, using Hit@1 for image and video tasks and NDCG@5 for visual documents. Therefore, the 80.6 score is a composite across these metrics, not a simple "80.6% of queries succeeded."

On MMEB-v3, which covers 190 tasks, the three versions score 56.0, 58.2, and 59.5. The 2B model's text and Agent subscores are 45.3 and 45.1. The 4B model's 58.2 beats various 7B-to-8B models in the same table, whose scores range from 26.3 to 47.1. The 9B model achieves 48.8 for text, 51.0 for Agent, and 49.3 for MCMR. Across the results tables, 9B has the highest overall scores on both MMEB-v2 and MMEB-v3.

Across 26 internal WeChat offline tasks, the 2B model averages 72.0, compared to 60.9 for the reference Qwen model. Additionally, 14 A/B tests in production all showed improvements and were fully rolled out, covering Channels, Official Accounts, Moments, and e-commerce.

Matryoshka vectors let you trim from 2048 down to 256 dimensions

Matryoshka allows the long vectors output by a single model to be truncated to fit a given budget.

Using 256 dimensions cuts the raw storage and dot-product computation scale to one-eighth of the full 2048 dimensions.

The 2B model defaults to 2048-dimensional vectors but also supports prefix lengths of 64, 128, 256, 512, 1024, and 1536. After taking the first N dimensions and re-applying L2 normalization, the result can be used as a shorter vector without retraining a separate model for each length.

On the image and video subsets of MMEB-v2, a 256-dimensional vector retains 98.7% of the performance of the full vector. With FP16, each dimension takes 2 bytes: a 2048-dimensional vector uses 4096 bytes, while a 256-dimensional one uses 512 bytes. For a billion vectors, that brings raw capacity down from 4.096 TB to 512 GB.

These calculations cover only the raw vectors, not ANN indexes, IDs, metadata, replicas, quantization, or hot/cold tiering. Accuracy loss is more noticeable at 64 and 128 dimensions, and visual documents are more sensitive to truncation. In practice, you might use 256 dimensions for low-cost recall and then switch to longer vectors or a reranker for a small candidate set.

Choosing between the 2B, 4B, and 9B models

Picking the right model involves balancing throughput, VRAM, index cost, and retrieval quality.

Start with your business scale and quality targets, then decide on model size and vector dimension.

The 2B model suits high-concurrency, high-volume, cost-sensitive recall systems. The 4B version is for scenarios that need better quality but can't justify the 9B's cost. The 9B model is best for quality-first retrieval, or for use as a teacher model and in the reranking stage. All three can use Matryoshka to adjust vector length.

Base dependencies can be installed via requirements.txt. For inference, you can use Transformers 5.2.0 or Sentence Transformers, with the latter offering a unified encoding interface for queries and documents and letting you directly choose the truncation dimension. For serving, vLLM 0.27.0 supports the pooling mode, and SGLang 0.5.9 supports the embedding mode; video serving requires additional patches.

The MMEB-v3 download, encoding, and scoring pipeline is provided, with video evaluation sampling 64 frames by default. Tencent's code and model weights are under Apache 2.0, but you'll need to check licenses separately for the underlying Qwen3.5, datasets, and third-party components. The current release includes model weights, inference, serving, and evaluation, but not the full training data, compute, or complete training pipeline.

For practical selection, start with 2B and 256 dimensions to establish a baseline, then compare 512 dimensions, full dimensions, and the 4B and 9B models in turn. Only when retrieval quality, batch throughput, index cost, and stability all meet your requirements on your own business data is the model truly production-ready.