Big Data Analytics with Datalog Queries on Spark.

Journal: Proceedings. ACM-SIGMOD International Conference on Management of Data
Published Date: Jan 1, 2016

Abstract

There is great interest in exploiting the opportunity provided by cloud computing platforms for large-scale analytics. Among these platforms, Apache Spark is growing in popularity for machine learning and graph analytics. Developing efficient complex analytics in Spark requires deep understanding of both the algorithm at hand and the Spark API or subsystem APIs (e.g., Spark SQL, GraphX). Our BigDatalog system addresses the problem by providing concise declarative specification of complex queries amenable to efficient evaluation. Towards this goal, we propose compilation and optimization techniques that tackle the important problem of efficiently supporting recursion in Spark. We perform an experimental comparison with other state-of-the-art large-scale Datalog systems and verify the efficacy of our techniques and effectiveness of Spark in supporting Datalog-based analytics.

Authors

  • Alexander Shkapsky
    University of California, Los Angeles.
  • Mohan Yang
    University of California, Los Angeles.
  • Matteo Interlandi
    University of California, Los Angeles.
  • Hsuan Chiu
    University of California, Los Angeles.
  • Tyson Condie
    University of California, Los Angeles.
  • Carlo Zaniolo
    University of California, Los Angeles.

Keywords

No keywords available for this article.

External Resources

Popular Topics
Recent Journals