[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
✨✨Latest Advances on Multimodal Large Language Models
A 100M-parameter multilingual TTS model for real-time CPU inference, voice cloning, and 48 kHz stereo generation
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.
Algorithms and Publications on 3D Object Tracking
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Parsing-free RAG supported by VLMs
[CVPR 2025 Highlight] Official code for "Olympus: A Universal Task Router for Computer Vision Tasks"
The open source implementation of Gemini, the model that will "eclipse ChatGPT" by Google
[CVPR 2025] MINIMA: Modality Invariant Image Matching
This repo contains the official code of our work SAM-SLR which won the CVPR 2021 Challenge on Large Scale Signer Independent Isolated Sign Language Recognition.
[ESSD 2025 & IEEE DFC 2025 & CVPRW 2026] Bright: A globally distributed multimodal VHR dataset for all-weather disaster response
An official PyTorch implementation of the CRIS paper
Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!
[CVPR 2023] Collaborative Diffusion
(NeurIPS 2022 CellSeg Challenge - 1st Winner) Open source code for "MEDIAR: Harmony of Data-Centric and Model-Centric for Multi-Modality Microscopy"
Long-RL: Scaling RL to Long Sequences (NeurIPS 2025)
Official repository for VisionZip (CVPR 2025)
To associate your repository with the multi-modality topic, visit your repo's landing page and select "manage topics."