webget.rb

Path: webget.rb
Last Update: Sun Oct 07 22:14:27 +1000 2007

Synopsis

This class provides your program functions to access WWW documents via HTTP, Hyper Text Transfer Protocol nversion 1.1., providing authorisation mechanisms (base 64), proxy access and static cookies Note: Does not follow 302 redirects by default.

Examples

GET+print

 require 'broswer/webget'
 agent = Browser::WebGet.new('http', 'www.example.com')
 agent.get('index.html')
 agent.print_response

GET+modify headers

 require 'broswer/webget'
 agent = Browser::WebGet.new('http', 'www.example.com')
 agent.get('index.html', {'User-Agent'=>'Customised header'})
 agent.print_response

Posting Form Data

1: Post+modify headers over SSL

 require 'broswer/webget'

 agent = Browser::WebGet.new('https', 'www.example.com') # or agent = Browser::WebGet.new('https', 'www.example.com', 443)
 agent.post('index.php', {'User-Agent'=>'Customised header'}, 'uname=jordan&passwd=123')
 agent.print_response

 Note: All posts will automatically add the appropriate post headers

Using a Proxy

 require 'broswer/webget'

1: GET+Using Proxy

 agent = Browser::WebGet.new('http', 'www.example.com', 80, '127.0.0.1', 8080)
 agent.get('index.html')

2: GET+Using Proxy with proxy basic authorisiation

 agent = Browser::WebGet.new('http', 'www.example.com', 80, '127.0.0.1', 8080)
 agent.proxy_auth('username', 'password')
 agent.get('index.html')

Using a Authentication

 require 'broswer/webget'

1: GET+Using basic authentication over SSL on high port

 agent = Browser::WebGet.new('https', 'www.example.com', 10443)
 agent.basic_auth('username', 'password')
 agent.get('index.html')

2: POST+over SSL+high port+with cookie

 agent = Browser::WebGet.new('https', 'www.example.com',10443,nil,nil,'sessionid=1234;nonce=4321')
 agent.basic_auth('username', 'password')
 agent.post('index.html',{},'hello=world')

 Note: If you need to use an authorisation method that is not supported then tunnel these
 requests through a proxy that can...

Viewing the response

 require 'broswer/webget'

 agent = Browser::WebGet.new('http', 'www.example.com')
 agent.get('index.html')
 p "200 OK" if agent.resp_header.code =~ /200/ or agent.resp_header.message =~/OK/
 p "Supports HTTP 1.1" if agent.resp_header.http_version =~ /1.1/
 p "Set Cookie Found!" if agent.resp_header.has_key("set-cookie")
 p "HTML form!" if agent.resp_body =~ /<HTML>/i

Setting SSL Mode

 require 'broswer/webget'

 agent = Browser::WebGet.new('https', 'www.example.com')
 agent.ssl_verify_mode = OpenSSL::SSL::VERIFY_PEER
 p agent.ssl_verify_mode?
 agent.ssl_verify_mode = OpenSSL::SSL::VERIFY_NONE

Author

Jordan Del-Grande, Security Technology Science Pty Ltd

Copyright

Copyright (c) 2007 by Security Technology Science Pty Ltd. All rights reserved. Licensed under the same terms as GNU GPL v2

Required files

net/https   openssl  

[Validate]