Please note that as of October 24, 2014, the Nokia Developer Wiki will no longer be accepting user contributions, including new entries, edits and comments, as we begin transitioning to our new home, in the Windows Phone Development Wiki. We plan to move over the majority of the existing entries. Thanks for all your past and future contributions.

Archived:How to use urllib with proxy in PySymbian

From Wiki
Jump to: navigation, search

Archived.pngArchived: This article is archived because it is not considered relevant for third-party developers creating commercial solutions today. If you think this article is still relevant, let us know by adding the template {{ReviewForRemovalFromArchive|user=~~~~|write your reason here}}.

Article Metadata
Tested with
Devices(s): E71
Platform(s): S60 3rd Edition
S60 3rd Edition (initial release)
Keywords: urllib, proxy
Created: marcelobarrosalmeida (May, 1 2009)
Last edited: hamishwillee (19 Jun 2012)


Nowadays proxy support has become a requisite for practically all applications that connect to the Internet. In general, they are used in intranets due to reasons like security, access control and bandwidth saving. If you are developing networked applications using urllib and PySymbian, it is interesting to add proxy support for it, generating a larger number of potential users.

In this article is presented a class called UrllibProxy that will help you in this task. The presented code will add proxy support and basic HTTP authentication for urllib. For instance, if you are behind a proxy, the following code will fail:


With few modifications, it is possible to download the desired URL, just adding proxy support:

proxy = "http://username:password@proxy_ip:proxy_port"
urlprx = UrllibProxy(proxy)

Proxies are specified using the string http://username:password@proxy_ip:proxy_port:

  • username: your username, if any
  • password: your password, if any
  • proxy_ip: proxy ip or name
  • proxy_port: proxy port

When user authentication is not required by proxy, use http://proxy_ip:proxy_port.

Moreover, if the site you want to access is protected with HTTP basic authentication, just add your credentials to UrllibProxy:

proxy = "http://username:password@proxy_ip:proxy_port"
urlprx = UrllibProxy(proxy,"site_login","site_password")


# -*- coding: utf-8 -*-
# Marcelo Barros de Almeida
# marcelobarrosalmeida (at)
import sys
# Try to import 'btsocket' as 'socket' - ignored on versions < 1.9.x
sys.modules['socket'] = __import__('btsocket')
except ImportError:
import socket
import urllib
from urllib import unquote, splittype, splithost
__all__ = [ "UrllibProxy" ]
class _FancyURLopener(urllib.FancyURLopener):
""" This class handles basic auth, providing user and password
when required by twitter

def __init__(self, usr, pwd, prx={}):
""" Set default values for local proxy (if any)
and set user/password for twitter

self.usr = usr
self.pwd = pwd
def prompt_user_passwd(self, host, realm):
""" Basic auth callback

return (self.usr,self.pwd)
class UrllibProxy(object):
""" Simple class for fetching URLs via urllib.
It adds proxy support and http basic authentication.

def __init__(self, proxy="", usr="", pwd=""):
""" Access a given url using proxy and possible http basic auth.
proxy  : standard proxy string, like http://username:password@proxy_ip:proxy_port
For proxies without authentication, just use http://proxy_ip:proxy_port
usr,pwd: if the site you want to access required basic HTTP authentication, like twitter,
provide user and password for this site. This authetication is not related to
the previous proxy authentication.

self.proxy = proxy
self.url = ""
self.usr, self.pwd = usr, pwd
self.user_agent = "urllib/1.0 (urllib)"
def _prepare_urlopener(self):
""" Update twitter status
Inspired in

if self.proxy:
XXX, r_type = splittype(self.proxy)
phost, XXX = splithost(r_type)
puser_pass = None
if '@' in phost:
user_pass, phost = phost.split('@', 1)
if ':' in user_pass:
user, password = user_pass.split(':', 1)
puser_pass = ('%s:%s' %
self.urlopener_proxy = {'http':'http://%s'%phost}
if not puser_pass:
self.headers = [('User-agent', self.user_agent)]
self.headers = [('User-agent', self.user_agent),
('Proxy-authorization', 'Basic ' + puser_pass) ]
self.urlopener_proxy = {}
self.headers = []
def urlopener(self):
""" Return an urlopener with authentication headers and proxy already set

urlopener = _FancyURLopener(self.usr, self.pwd, self.urlopener_proxy)
urlopener.addheaders = self.headers
return urlopener
def open(self,url,params=""):
self.url = url
if params:
f = self.urlopener().open(self.url,params) #post
f = self.urlopener().open(self.url) #get
return f
def urlretrieve(self, url, filename=None):
if not filename:
filename = url[url.rfind("/") + 1:]
f = open(filename,"wb")
if __name__ == "__main__":
local_file = "e:\\"
url = ""
proxy = "http://username:password@"
urlprx = UrllibProxy(proxy)
urlprx.urlretrieve(url, local_file)


This page was last modified on 19 June 2012, at 01:21.
63 page views in the last 30 days.