spider.rb

Path: lib/crawler/spider.rb
Last Update: Tue Jan 08 19:49:06 +1100 2008

Synopsis

This class crawls a given URL restricting further requests to the URL domain Capable of parsing:

  1. html links (anchor tags, img tags)
  2. frames (frame tags, iframe tags)
  3. form fields building GET/POST requests with query strings

Copyright

Copyright (c) 2007 - 2008 by Security Technology Science Pty Ltd. All rights reserved.

License

This file is part of STS Scanner.

STS Scanner is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

STS Scanner is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with STS Scanner. If not, see <www.gnu.org/licenses/>.

Required files

logger  

[Validate]