DeepSeek-VL

Florence-2-large is an advanced vision foundation model developed by Microsoft, capable of handling a wide variety of vision and vision-language tasks, such as captioning, object detection, segmentation, and OCR. Built with a sequence-to-sequence architecture, it uses the FLD-5B dataset containing over 5 billion annotations and 126 million images to master multi-task learning. Florence-2-large excels in both zero-shot and fine-tuned settings, providing high-quality results with minimal training. The model supports tasks including detailed captioning, object detection, and dense region captioning, and can process images with text prompts to generate relevant responses. It offers great flexibility by handling diverse vision-related tasks through prompt-based approaches, making it a competitive tool in AI-powered visual tasks. The model is available on Hugging Face with pre-trained weights, enabling users to quickly get started with image processing and task execution.

LLaVA (Large Language-and-Vision Assistant) is an innovative multimodal model that integrates a vision encoder with the Vicuna language model to facilitate comprehensive visual and language understanding. Through end-to-end training, LLaVA exhibits impressive chat capabilities, emulating the multimodal functionalities of models like GPT-4. Notably, LLaVA-1.5 has achieved state-of-the-art performance across 11 benchmarks, utilizing publicly available data and completing training in approximately one day on a single 8-A100 node, surpassing methods that rely on billion-scale datasets. The development of LLaVA involved the creation of a multimodal instruction-following dataset, generated using language-only GPT-4. This dataset comprises 158,000 unique language-image instruction-following samples, including conversations, detailed descriptions, and complex reasoning tasks. This data has been instrumental in training LLaVA to perform a wide array of visual and language tasks effectively.