Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities
In brief
- A single parser handles documents, images, audio, and video using one encoder and one schema.
- Record scores on OmniDocBench v1.6 (96.96) and OmniParsingBench (75.08) with a 5B-parameter model.
- Model code and weights are released under open-source license.
Researchers unveil Youtu-Parsing-Omni, an omni-modal parser that converts documents, images, audio, and video into structured records anchored in space and time. According to the announcement, the model uses a single encoder per modality and a unified schema for all tasks, embedding cross-modal fusion and temporal alignment directly into perception.
Trained with a self-evolving post-training procedure called OSAD, the system achieves state-of-the-art results on OmniDocBench v1.6 and OmniParsingBench, with a clear edge on audio and video data. It also outperforms specialized models in table extraction and music transcription while remaining open-source, with code and weights publicly available.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error