View a PDF of the paper titled CLIBD: Bridging Imaginative and prescient and Genomics for Biodiversity Monitoring at Scale, by ZeMing Gong and 6 different authors
Summary:Measuring biodiversity is essential for understanding ecosystem well being. Whereas prior works have developed machine studying fashions for taxonomic classification of photographic photographs and DNA individually, on this work, we introduce a multimodal method combining each, utilizing CLIP-style contrastive studying to align photographs, barcode DNA, and text-based representations of taxonomic labels in a unified embedding house. This permits for correct classification of each identified and unknown insect species with out task-specific fine-tuning, leveraging contrastive studying for the primary time to fuse DNA and picture knowledge. Our technique surpasses earlier single-modality approaches in accuracy by over 8% on zero-shot studying duties, showcasing its effectiveness in biodiversity research.
Submission historical past
From: ZeMing Gong [view email]
[v1]
Mon, 27 Could 2024 17:57:48 UTC (5,008 KB)
[v2]
Thu, 31 Oct 2024 20:07:53 UTC (41,938 KB)
Source link
#Bridging #Imaginative and prescient #Genomics #Biodiversity #Monitoring #Scale
