Welcome to MapReduce Design Patterns! This book will be unique in some ways and
familiar in others. First and foremost, this book is obviously about design patterns, which
are templates or general guides to solving problems. We took a look at other design
patterns books that have been written in the past as inspiration, particularly Design
Patterns: Elements of Reusable Object-Oriented Software, by Gamma et al. (1995), which
is commonly referred to as “The Gang of Four” book. For each pattern, you’ll see a
template that we reuse over and over that we loosely based off of their book. Repeatedly
seeing a similar template will help you get to the specific information you need. This
will be especially useful in the future when using this book as a reference.
This book is a bit more open-ended than a book in the “cookbook” series of texts as we
don’t call out specific problems. However, similarly to the cookbooks, the lessons in this
book are short and categorized. You’ll have to go a bit further than just copying and
pasting our code to solve your problems, but we hope that you will find a pattern to get
you at least 90% of the way for just about all of your challenges.
This book is mostly about the analytics side of Hadoop or MapReduce. We intentionally
try not to dive into too much detail on how Hadoop or MapReduce works or talk too
long about the APIs that we are using. These topics have been written about quite a few
times, both online and in print, so we decided to focus on analytics.
In this preface, we’ll talk about how to read this book since its format might be a bit
different than most books you’ve read.
Until now, design patterns for the MapReduce framework have been scattered among various research papers, blogs, and books. This handy guide brings together a unique collection of valuable MapReduce patterns that will save you time and effort regardless of the domain, language, or development framework you’re using.
Each pattern is explained in context, with pitfalls and caveats clearly identified to help you avoid common design mistakes when modeling your big data architecture. This book also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop.
Summarization patterns: get a top-level view by summarizing and grouping data
Filtering patterns: view data subsets such as records generated from one user
Data organization patterns: reorganize data to work with other systems, or to make MapReduce analysis easier
Join patterns: analyze different datasets together to discover interesting relationships
Metapatterns: piece together several patterns to solve multi-stage problems, or to perform several analytics in the same job
Input and output patterns: customize the way you use Hadoop to load or store data
"A clear exposition of MapReduce programs for common data processing patterns—this book is indespensible for anyone using Hadoop."
--Tom White, author of Hadoop: The Definitive Guide