DeepDive: Declarative Knowledge Base Construction.

Journal: SIGMOD record
Published Date: Feb 6, 2016

Abstract

The dark data extraction or knowledge base construction (KBC) problem is to populate a SQL database with information from unstructured data sources including emails, webpages, and pdf reports. KBC is a long-standing problem in industry and research that encompasses problems of data extraction, cleaning, and integration. We describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems. The key idea in DeepDive is that statistical inference and machine learning are key tools to attack classical data problems in extraction, cleaning, and integration in a unified and more effective manner. DeepDive programs are declarative in that one cannot write probabilistic inference algorithms; instead, one interacts by defining features or rules about the domain. A key reason for this design choice is to enable domain experts to build their own KBC systems. We present the applications, abstractions, and techniques of DeepDive employed to accelerate construction of KBC systems.

Authors

  • Christopher De Sa
    Departments of Electrical Engineering and Computer Science, Stanford University.
  • Alex Ratner
    Stanford University, Stanford, CA.
  • Christopher Ré
    1Stanford University, Stanford, CA USA.
  • Jaeho Shin
    Stanford University.
  • Feiran Wang
    Stanford University.
  • Sen Wu
    1Stanford University, Stanford, CA USA.
  • Ce Zhang
    Stanford University.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals