Home | Amazing | Today | Tags | Publishers | Years | Account | Search 
Spark for Python Developers

Spark for Python Developers, 9781784399696 (1784399698), Packt Publishing, 2015

Key Features

  • Set up real-time streaming and batch data intensive infrastructure using Spark and Python
  • Deliver insightful visualizations in a web app using Spark (PySpark)
  • Inject live data using Spark Streaming with real-time events

Book Description

Looking for a cluster computing system that provides high-level APIs? Apache Spark is your answer―an open source, fast, and general purpose cluster computing system. Spark's multi-stage memory primitives provide performance up to 100 times faster than Hadoop, and it is also well-suited for machine learning algorithms.

Are you a Python developer inclined to work with Spark engine? If so, this book will be your companion as you create data-intensive app using Spark as a processing engine, Python visualization libraries, and web frameworks such as Flask.

To begin with, you will learn the most effective way to install the Python development environment powered by Spark, Blaze, and Bookeh. You will then find out how to connect with data stores such as MySQL, MongoDB, Cassandra, and Hadoop.

You'll expand your skills throughout, getting familiarized with the various data sources (Github, Twitter, Meetup, and Blogs), their data structures, and solutions to effectively tackle complexities. You'll explore datasets using iPython Notebook and will discover how to optimize the data models and pipeline. Finally, you'll get to know how to create training datasets and train the machine learning models.

By the end of the book, you will have created a real-time and insightful trend tracker data-intensive app with Spark.

What you will learn

  • Create a Python development environment powered by Spark (PySpark), Blaze, and Bookeh
  • Build a real-time trend tracker data intensive app
  • Visualize the trends and insights gained from data using Bookeh
  • Generate insights from data using machine learning through Spark MLLIB
  • Juggle with data using Blaze
  • Create training data sets and train the Machine Learning models
  • Test the machine learning models on test datasets
  • Deploy the machine learning algorithms and models and scale it for real-time events

About the Author

Amit Nandi studied physics at the Free University of Brussels in Belgium, where he did his research on computer generated holograms. Computer generated holograms are the key components of an optical computer, which is powered by photons running at the speed of light. He then worked with the university Cray supercomputer, sending batch jobs of programs written in Fortran. This gave him a taste for computing, which kept growing. He has worked extensively on large business reengineering initiatives, using SAP as the main enabler. He focused for the last 15 years on start-ups in the data space, pioneering new areas of the information technology landscape. He is currently focusing on large-scale data-intensive applications as an enterprise architect, data engineer, and software developer. He understands and speaks seven human languages. Although Python is his computer language of choice, he aims to be able to write fluently in seven computer languages too.

Table of Contents

  1. Setting Up a Spark Virtual Environment
  2. Building Batch and Streaming Apps with Spark
  3. Juggling Data with Spark
  4. Learning from Data Using Spark
  5. Streaming Live Data with Spark
  6. Visualizing Insights and Trends
(HTML tags aren't allowed.)

Engineering Design Reliability Handbook
Engineering Design Reliability Handbook

Researchers in the engineering industry and academia are making important advances on reliability-based design and modeling of uncertainty when data is limited. Non deterministic approaches have enabled industries to save billions by reducing design and warranty costs and by improving quality.

Considering the lack of

Exhibiting Photography: A Practical Guide to Choosing a Space, Displaying Your Work, and Everything in Between
Exhibiting Photography: A Practical Guide to Choosing a Space, Displaying Your Work, and Everything in Between
This book originated in workshops taught initially at the University of Westminster and subsequently at Photofusion and the City Lit. The aim of the workshops was to empower students by opening up the processes and practices of exhibiting. What the workshops taught me was that, although students are increasingly working towards a career aim of...
Real-Life Math: Everyday Use of Mathematical Concepts
Real-Life Math: Everyday Use of Mathematical Concepts
"What does this have to do with real life?" is a question that plagues mathematics teachers across America, as students are confronted with abstract topics in their high school mathematics courses. The National Council of Teachers of Mathematics emphasizes the importance of making real world connections in teaching mathematics so that...

Practical Neo4j
Practical Neo4j

Why have developers at places like Facebook and Twitter increasingly turned to graph databases to manage their highly connected big data? The short answer is that graphs offer superior speed and flexibility to get the job done.

It’s time you added skills in graph databases to your toolkit....

Accounting: Concepts and Applications
Accounting: Concepts and Applications

No matter what your career plans or future goals, ACCOUNTING: CONCEPTS AND APPLICATIONS, 10e helps you develop a solid understanding of accounting and its importance in business today that will put you well ahead of the competition. Organized around business activities, the text balances an introduction to accounting procedures with an...

Computer-Enhanced and Mobile-Assisted Language Learning: Emerging Issues and Trends
Computer-Enhanced and Mobile-Assisted Language Learning: Emerging Issues and Trends
Since the publication of the Handbook of Research on Computer-Enhanced Language Acquisition and Learning in 2008, information communication technology (ICT) has continued to create new learning paths to assist language learning. While CD-ROMs, multimedia computer labs, the World Wide Web, e-mail, and SMS still play an important...
©2019 LearnIT (support@pdfchm.net) - Privacy Policy