Home | Amazing | Today | Tags | Publishers | Years | Account | Search 
Web Scraping with Python (Community Experience Distilled)


Successfully scrape data from any website with the power of Python

About This Book

  • A hands-on guide to web scraping with real-life problems and solutions
  • Techniques to download and extract data from complex websites
  • Create a number of different web scrapers to extract information

Who This Book Is For

This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved.

What You Will Learn

  • Extract data from web pages with simple Python programming
  • Build a threaded crawler to process web pages in parallel
  • Follow links to crawl a website
  • Download cache to reduce bandwidth
  • Use multiple threads and processes to scrape faster
  • Learn how to parse JavaScript-dependent websites
  • Interact with forms and sessions
  • Solve CAPTCHAs on protected web pages
  • Discover how to track the state of a crawl

In Detail

The Internet contains the most useful set of data ever assembled, largely publicly accessible for free. However, this data is not easily reusable. It is embedded within the structure and style of websites and needs to be carefully extracted to be useful. Web scraping is becoming increasingly useful as a means to easily gather and make sense of the plethora of information available online. Using a simple language like Python, you can crawl the information out of complex websites using simple programming.

This book is the ultimate guide to using Python to scrape data from websites. In the early chapters it covers how to extract data from static web pages and how to use caching to manage the load on servers. After the basics we'll get our hands dirty with building a more sophisticated crawler with threads and more advanced topics. Learn step-by-step how to use Ajax URLs, employ the Firebug extension for monitoring, and indirectly scrape data. Discover more scraping nitty-gritties such as using the browser renderer, managing cookies, how to submit forms to extract data from complex websites protected by CAPTCHA, and so on. The book wraps up with how to create high-level scrapers with Scrapy libraries and implement what has been learned to real websites.

Style and approach

This book is a hands-on guide with real-life examples and solutions starting simple and then progressively becoming more complex. Each chapter in this book introduces a problem and then provides one or more possible solutions.

(HTML tags aren't allowed.)

Image Recognition and Classification (Optical Engineering, 78)
Image Recognition and Classification (Optical Engineering, 78)
This book presents important recent advances in sensors, image processing algorithms, and systems for image recognition and classification with diverse applications in military, aerospace, security, image tracking, radar, biomedical, and intelligent transportation. The book includes contributions by some of the leading researchers in the field to...
Mike and Phani's Essential C++ Techniques
Mike and Phani's Essential C++ Techniques

C++ is the language of choice for developing the most sophisticated Windows programs, but it is filled with hidden traps for the unwary. Mike Hyman and Phani Vaddadi's no-nonsense book helps C++ programmers avoid these traps by providing invaluable techniques gleaned from a combined 30 years of...

Algorithms Sequential and Parallel: A Unified Approach
Algorithms Sequential and Parallel: A Unified Approach

A major thrust of computer science is the design, analysis, implementation, and scicntific evaluation of algorithms to solve critical problems. In addition, new challenges are being offered to computer scientists in the field of computational science and engineering, which includes challenging problems in computational biology,...

UNIX, Third Edition (Visual QuickStart Guide)
UNIX, Third Edition (Visual QuickStart Guide)
Need to learn Unix fast? Try a Visual QuickStart! This best-selling reference's visual format and step-by-step, task-based instructions will have you up and running with Unix software in no time. In this completely updated edition of our best-selling guide to Unix, Second Edition: Visual...
RailsSpace: Building a Social Networking Website with Ruby on Rails (Addison-Wesley Professional Ruby Series)
RailsSpace: Building a Social Networking Website with Ruby on Rails (Addison-Wesley Professional Ruby Series)
Ruby on Rails is fast displacing PHP, ASP, and J2EE as the development framework of choice for discriminating programmers, thanks to its elegant design and emphasis on practical results. RailsSpace teaches you to build large-scale projects with Rails by developing a real-world application: a social networking website like...
Cloud Standards: Agreements That Hold Together Clouds
Cloud Standards: Agreements That Hold Together Clouds

Cloud computing is often described as providing computing resources the way electric utilities provide energy. In theory, anyone with an adequate connection to the Internet should be able to tap into a cloud provider and get exactly the computing resources they want when they want it, just like plugging into the electricity grid and...

©2018 LearnIT (support@pdfchm.net) - Privacy Policy