Introducing Crunch: Easy MapReduce Pipelines for Hadoop

From knowledgebase
Revision as of 10:40, 25 August 2020 by Dirkjan (talk | contribs) (Created page with "=URL= http://www.cloudera.com/blog/2011/10/introducing-crunch/ =Description= Today, we’re pleased to introduce Crunch, a Java library that aims to make writing, testing,...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

URL

http://www.cloudera.com/blog/2011/10/introducing-crunch/

Description

Today, we’re pleased to introduce Crunch, a Java library that aims to make writing, testing, and running MapReduce pipelines easy, efficient, and even fun. Crunch’s design is modeled after Google’s FlumeJava, focusing on a small set of simple primitive operations and lightweight user-defined functions that can be combined to create complex, multi-stage pipelines. At runtime, Crunch compiles the pipeline into a sequence of MapReduce jobs and manages their execution.

Retrieved from "https://wiki.dirkjanswagerman.nl/index.php?title=Introducing_Crunch:_Easy_MapReduce_Pipelines_for_Hadoop&oldid=3875"