Distributed Data Processing Frameworks

Distributed Data Processing Frameworks

• Mapreduce Framework
• Research Papers 
• MapReduce: Simplified Data Processing on Large Clusters
• Apache Hadoop Goes Realtime at Facebook
• Apache Hadoop
• Projects in Hadoop Family 
• Hive
• Pig
• HBase 
• Column-Oriented Database
• Apache Sqoop 
• Hadoop RDBMS
• Zookeeper
• Impala
• Massively parallel processing SQL for Hadoop
• Apache Accumulo
• Sorted, distributed key/value store; based on Google’s BigTable.
• Bulk Synchronous Parallel Model
• Apache Hama 
• Pregel
• Bulk Synchronous Parallel Computations for processing Graphs
• Research Paper 
• Pregel: A System for Large-Scale Graph Processing
• Apache Giraph  
• Stratosphere
• Research Paper 
• Nephele/PACTs: A Programming Model and Execution Framework for Web-Scale Analytical Processing
• Spark
• Spark Streaming, MLlib, GraphX, Shark
• Storm
• Distributed and fault-tolerant real time computation
• Percolator
• Research Paper 
• Large-scale Incremental Processing Using Distributed Transactions and Notifications
• Dremel
• Research Paper 
• Dremel: Interactive Analysis of Web-Scale Datasets