DeepRead
Cover of Webbots, spiders, and screen scrapers

Webbots, spiders, and screen scrapers

Michael Schrenk · 2006

Machine learning, statistics and probability › Applied machine learning and computer vision

#13,395 most recommended

About the book

A practical guide to writing programs that act as automated clients of the World Wide Web: bots, spiders, and screen scrapers that request pages, submit forms, handle cookies and sessions, and extract information from HTML. It is aimed at programmers who want to collect or monitor web data without a human at the browser, and it covers the web-programming techniques that work requires. The core position is that web pages are a usable programmatic data source, so software can retrieve and interpret them as reliably as it reads a file or database. The book presents the mechanics of doing this well, including parsing content that was designed for human readers. It also holds that such automation has to be built with care for the sites it visits, weighing legal and ethical limits such as site terms and robots exclusion conventions.

Written by Claude from what it knows of this book.

What Hacker News says 50% recommend · 0% critical

In the commenters’ own words

What Open Library readers say

No Open Library reader has rated it yet.

  • 0 want to read
  • 0 reading
  • 1 have read

Open Library’s page for the book

What critics say, via Wikipedia

No Wikipedia article about this book yet, so no critics to quote.

Reddit communities — coming later

Details

First published
2006
ISBN
9781593271206
Editions known
2 at Open Library
Open Library
OL8989112W

Intelligent agents (Computer software)Intelligent agents (computer software)Internet programmingInternet searchingWeb search engines

Explore in the mosaic

Read next: its neighbours on the mosaic