DeepSeek-OCR: Context Optical Compression —

DeepSeek just released DeepSeek-OCR — a revolutionary context optical compression model that transforms visual contexts into textual representations. Unlike traditional OCR tools that extract text from images, DeepSeek-OCR compresses entire visual scenes into dense, structured text suitable for LLMs, enabling multimodal understanding at unprecedented scale.

What is DeepSeek-OCR?

DeepSeek-OCR is a vision encoder designed from an LLM-centric perspective. Its core innovation is context optical compression — converting complex visual scenes into compact text representations that preserve spatial relationships, object properties, and semantic meaning. This enables LLMs to reason over visual data without needing native vision capabilities.

Built on top of a transformer architecture with Mamba layers for efficient long-range processing, DeepSeek-OCR handles images at native resolutions while compressing them into a fixed token budget. The result: a text-only representation of complex visual scenes that LLMs can process naturally.

Core Architecture

DeepSeek-OCR uses a hybrid transformer-Mamba architecture optimized for visual compression:

Resolution Modes

The model supports multiple resolution modes optimized for different use cases:

How It Works

DeepSeek-OCR processes images through three stages:

Implementation Options

DeepSeek-OCR offers multiple deployment options:

vLLM Inference (Recommended)

# Pull the container
docker run -it --rm --gpus all   nvcr.io/nim/nvidia/nemotron-3-nano-omni

# Or use the API endpoint
curl -X POST https://integrate.api.nvidia.com/v1/chat/completions   -H "Authorization: Bearer $NVIDIA_API_KEY"   -H "Content-Type: application/json"   -d '{
    "model": "nvidia/nemotron-3-nano-omni",
    "messages": [{"role": "user", "content": "Describe this image"}],
    "max_tokens": 512
  }'
    

Transformers Inference

from transformers import AutoModel, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained(
  "deepseek-ai/DeepSeek-OCR",
  trust_remote_code=True
)
model = AutoModel.from_pretrained(
  "deepseek-ai/DeepSeek-OCR",
  _attn_implementation='flash_attention_2',
  trust_remote_code=True,
  use_safetensors=True
)
model = model.eval().cuda().to(torch.bfloat16)

prompt = "\n<|grounding|>Convert the document to markdown."
res = model.infer(
  tokenizer,
  prompt=prompt,
  image_file='your_image.jpg',
  output_path='your/output/dir',
  base_size=1024,
  image_size=640,
  crop_mode=True,
  save_results=True,
  test_compress=True
)
    

Ollama

ollama run deepseek-ocr

Unsloth

pip install unsloth
unsloth cli download deepseek-ai/DeepSeek-OCR

Prompt Examples

DeepSeek-OCR supports multiple prompt modes:

• Document OCR: "<image>\n<|grounding|>Convert the document to markdown."
• Image Description: "<image>\n<|grounding|>OCR this image."
• Free OCR: "<image>\nFree OCR."
• Figure Parsing: "<image>\nParse the figure."
• General Description: "<image>\nDescribe this image in detail."
• Grounding: "<image>\nLocate <|ref|>xxxx<|/ref|> in the image."

Use Cases

Technical Requirements

Key Metrics

Resources

• GitHub Repository
• Hugging Face Model
• Arxiv Paper
• vLLM Documentation