GAIA - A Multi-media Multi-lingual Knowledge Extraction and Hypothesis Generation System

Authors

Tongtao Zhang, Ananya Subburathinam, Ge Shi, Lifu Huang, Di Lu, Xiaoman Pan, Manling Li, Boliang Zhang, Qingyun Wang, Spencer Whitehead, Heng Ji, Alireza Zareian, Hassan Akbari, Brian Chen, Ruiqi Zhong, Steven Shao, Emily Allaway, Shih-Fu Chang, Kathleen McKeown Dongyu Li, Xin Huang, Kexuan Sun, Xujun Peng, Ryan Gabbard, Marjorie Freedman,Mayank Kejriwal, Ram Nevatia, Pedro Szekely, T. K. Satish Kumar, Ali Sadeghian, Giacomo Bergami, Sourav Dutta, Miguel Rodriguez, Daisy Zhe Wang

Abstract

An analyst or a planner seeking a rich, deep under-standing of an emergent situation today is faced with a paradox - multimodal, multilingual real-time information about most emergent situationsis freely available but the sheer volume and diver-sity of such information makes the task of under-standing a specific situation or finding relevant in-formation an immensely challenging one. To remedy this situation, the Generating Alternative Interpretations for Analysis (GAIA) team at DARPA AIDA program aims for automated solutions that provide an integrated, comprehensive, nuanced, and timely view of emerging events, situations,and trends of interest. GAIA focuses on developing a multi-hypothesis semantic engine that embodies a novel synthesis of new and existing technologies in multimodal knowledge extraction, semantic integration, knowledge graph generation, and inference.