Ophthalmology

ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement

We present ILLUME+ that leverages dual visual tokenization and a diffusion decoder to improve both deep semantic understanding and high-fidelity image generation. Existing unified models have struggled to simultaneously handle the three fundamental capabilities in a unified model: understanding, generation, and editing. Models like Chameleon and EMU3 utilize VQGAN for image discretization, due t...

Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness

The rapid development of Large Multimodal Models (LMMs) for 2D images and videos has spurred efforts to adapt these models for interpreting 3D scenes. However, the absence of large-scale 3D vision-language datasets has posed a significant obstacle. To address this issue, typical approaches focus on injecting 3D awareness into 2D LMMs by designing 3D input-level scene representations. This work p...

Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images

Artificial Intelligence (AI) in skin disease diagnosis has improved significantly, but a major concern is that these models frequently show biased p...

UniViTAR: Unified Vision Transformer with Native Resolution

Conventional Vision Transformer simplifies visual modeling by standardizing input resolutions, often disregarding the variability of natural visual ...

STPNet: Scale-aware Text Prompt Network for Medical Image Segmentation

Accurate segmentation of lesions plays a critical role in medical image analysis and diagnosis. Traditional segmentation approaches that rely solely...

Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks

Vision-Language Models (VLMs) extend the capabilities of Large Language Models (LLMs) by incorporating visual information, yet they remain vulnerabl...

A Retina-Inspired Pathway to Real-Time Motion Prediction inside Image Sensors for Extreme-Edge Intelligence

The ability to predict motion in real time is fundamental to many maneuvering activities in animals, particularly those critical for survival, such ...

TDBench: Benchmarking Vision-Language Models in Understanding Top-Down Images

The rapid emergence of Vision-Language Models (VLMs) has significantly advanced multimodal understanding, enabling applications in scene comprehensi...

Machine Learning Prediction of Cardiovascular Risk in Type 1 Diabetes Mellitus Using Radiomics Features from Multimodal Retinal Images

This study aimed to develop a machine learning (ML) algorithm capable of determining cardiovascular risk in multimodal retinal images from patients ...

SViQA: A Unified Speech-Vision Multimodal Model for Textless Visual Question Answering

Multimodal models integrating speech and vision hold significant potential for advancing human-computer interaction, particularly in Speech-Based Vi...

Scaling Language-Free Visual Representation Learning

Visual Self-Supervised Learning (SSL) currently underperforms Contrastive Language-Image Pretraining (CLIP) in multimodal settings such as Visual Qu...

QSViT: A Methodology for Quantizing Spiking Vision Transformers

Vision Transformer (ViT)-based models have shown state-of-the-art performance (e.g., accuracy) in vision-based AI tasks. However, realizing their ca...

CellVTA: Enhancing Vision Foundation Models for Accurate Cell Segmentation and Classification

Cell instance segmentation is a fundamental task in digital pathology with broad clinical applications. Recently, vision foundation models, which ar...

ToVE: Efficient Vision-Language Learning via Knowledge Transfer from Vision Experts

Vision-language (VL) learning requires extensive visual perception capabilities, such as fine-grained object recognition and spatial perception. Rec...

QG-VTC: Question-Guided Visual Token Compression in MLLMs for Efficient VQA

Recent advances in Multi-modal Large Language Models (MLLMs) have shown significant progress in open-world Visual Question Answering (VQA). However,...

Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features

Visual token reduction lowers inference costs caused by extensive image features in large vision-language models (LVLMs). Unlike relevant studies th...

Training Frozen Feature Pyramid DINOv2 for Eyelid Measurements with Infinite Encoding and Orthogonal Regularization

Accurate measurement of eyelid parameters such as Margin Reflex Distances (MRD1, MRD2) and Levator Function (LF) is critical in oculoplastic diagnos...

Enhancing Fundus Image-based Glaucoma Screening via Dynamic Global-Local Feature Integration

With the advancements in medical artificial intelligence (AI), fundus image classifiers are increasingly being applied to assist in ophthalmic diagn...

Beyond Wide-Angle Images: Unsupervised Video Portrait Correction via Spatiotemporal Diffusion Adaptation

Wide-angle cameras, despite their popularity for content creation, suffer from distortion-induced facial stretching-especially at the edge of the le...

Integrating electrocardiogram and fundus images for early detection of cardiovascular diseases

Cardiovascular diseases (CVD) are a predominant health concern globally, emphasizing the need for advanced diagnostic techniques. In our research, w...