Model Releases

JD open-sources real-time video vision-language interaction model JoyAI-VL-Interaction, moving LLMs from Q&A to watching and speaking

AI summary

JD officially announced the open-sourcing of JoyAI-VL-Interaction, a real-time video vision-language interaction model. It is the world's first fully open-source interaction model and system, with day-0 native support from vLLM-Omni. The model has three breakthroughs: proactive judgment (continuously observing the video stream and deciding on its own when to speak), real-time response (responding instantly to ongoing video streams), and timely agent delegation (handing complex tasks to background models while the foreground continues observing). It supports video inputs such as cameras, live streams and surveillance streams, as well as voice input and output, a visual interface, long-term memory and vLLM deployment. In 58 human blind evaluation cases, it achieved an overall win rate of 77.6% against Doubao's video call assistant and 87.9% against Gemini's video call assistant.

Key Takeaways

  • JD officially announced the open-sourcing of JoyAI-VL-Interaction,a real-time video…
  • It is the world's first fully open-source interaction model and system,with day-0 native support…
  • The model has three breakthroughs: proactive judgment (continuously observing the video stream and…
  • It supports video inputs such as cameras,live streams and surveillance streams,as well as voice…
  • In 58 human blind evaluation cases,it achieved an overall win rate of 77.

Quick Facts

Published
Updated
Source
IT之家·人工智能
Category
Model Releases
AI score
71
Read the original Read the full article at the original source
Why it matters

京东开源了首个全栈实时视频交互模型,让大模型能持续观察并主动响应,开发者可以直接用代码搭建安防、导购等实时AI助手,实用性很强。

Related entities

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: JD open-sources real-time video vision-language interaction model JoyAI-VL-Interaction, moving LLMs from Q&A to watching and speaking. June 22, 2026. https://new.nat6.net/en/news/cmqozph2l05nlslx6ip3k2irx/

View all →