

    WebFetcher

    Niklas Frykholm (niklas@kagi.com)
    0.5.5, 08 Jul 2004


WHAT IS WEBFETCHER?

    WebFetcher is a module designed to facilitate fetching large
    ammounts of documents and images from the web. With WebFetcher it
    is easy to do such things as downloading all the images on a page.
    You can also do more complex tasks, such as fetching all the pages
    linked to by a certain page and all the images on those pages. You
    can save the documents in a tree structure mirroring the server
    layout or flat in a single directory. In either case, WebFetcher
    can translate the links on the pages so that all relationships are
    preserved.


SAMPLE USAGE

    Save a copy of Programming Ruby to your hard drive:

        require 'webfetcher'
        
        book = WebFetcher::Page.url('http://www.rubycentral.com/book/')
        book.recurse.save('pickaxe')


HOW TO GET WEBFETCHER

    The latest version of WebFetcher is available at
    http://www.acc.umu.se/~r2d2/programming/ruby/webfetcher/.


INSTALLATION

    Run

        ruby install.rb

    or copy the file webfetcher.rb to the desired destination manually.


VERSION HISTORY

    * 0.5.5 Bug fix for ruby 1.8.1. Better relative url handling.
         (Changes contributed by Xiangrong Fang, xrfang@hotmail.com.)
    * 0.5.4 Bug fix for ruby 1.6.4.
    * 0.5.3 Bug fix for ruby 1.6.6. Some cookie support.
    * 0.5.2 Workaround for block_given? and support for #-links.
    * 0.5.1 Now works on ruby 1.6 (previously only on 1.7).
    * 0.5.0 First release.


LICENSE

    WebFetcher is released under the same license as Ruby.


THE AUTHOR

    Feel free to send comments, bug reports and feature requests to the
    author. Here is the author's latest email address.

    niklas@kagi.com


MORE

    More documentation is found in the file webfetcher.html. In
    examples.rb you find some sample scripts.
