×
Namespaces

Variants
Actions

Archived:Python on Symbian/09. Basic Network Programming

From Nokia Developer Wiki
Jump to: navigation, search

Archived.pngAquivado: Este artigo foi arquivado, pois o conteúdo não é mais considerado relevante para se criar soluções comerciais atuais. Se você achar que este artigo ainda é importante, inclua o template {{ForArchiveReview|escreva a sua justificativa}}.

All PySymbian articles have been archived. PySymbian is no longer maintained by Nokia and is not guaranteed to work on more recent Symbian devices. It is not possible to submit apps to Nokia Store.

Article Metadata
Code ExampleArticle
Created: hamishwillee (30 Nov 2010)
Last edited: hamishwillee (08 May 2013)

Original Author: Marcelo Barros

This chapter provides an overview of basic IP networking concepts, including how to use the Python sockets API to set up ad-hoc, multicast and broadcast networks.

Contents

Introduction

Networked applications have fundamentally changed the way that people interact and perform their day to day tasks; we take it for granted that we can communicate, play, bank, collaborate, search for and share information, shop, and so on, without even needing to leave our computers. Mobile networked applications offer a further paradigm shift: not only can we do all of these things wherever we are, but we can share our location, presence and preferences to allow these (and other) services to offer us a much more personal and tailored experience. The mobile phone is already the most personal device that most people own; networked applications have the opportunity to make it even more so.

Python is a great runtime for writing networked applications, making it possible to construct network servers and clients with just few lines of code. Python on Symbian shares this benefit; there are a few different paradigms associated with mobile devices (and consequently some different socket options) but, in general, porting from the desktop or writing network code from scratch is a straightforward exercise.

This chapter is accompanied by worked examples showing how to set up simple ad-hoc networks with a client and server, and how to use multicast and broadcast networking. In addition, the chapter briefly covers how to use exception handling and other debugging techniques in your networked application.

Basic Network Principles

Network programming is difficult if you don't have at least a basic knowledge of IP addressing. The following sections discuss IP addressing schemes and unicast, multicast and broadcast addresses.

There are two main addressing schemes in use at time of writing: IPv4 (Internet Protocol version 4) and IPv6 (Internet Protocol version 6). IPv4 is by far the most common scheme used today, and is the one we'll discuss in this chapter.

IPv4 Addressing

There are three types of addresses in IPv4: unicast, multicast and broadcast:

  • Unicast is the normal point-to-point connection between two devices on a network. Several protocols uses unicast addressing, including HTTP (Internet browsing), SMTP (email) and FTP (file transfer).
  • Multicast is used to address messages to a specific group of machines, rather than a specific machine address. Machines that want to belong to a group must do it explicitly, joining the group via a socket API call. The group is, in fact, an IP address from a reserved range, as discussed later.
  • Broadcast addresses are similar to multicast but less selective: messages are sent to all machines (no subscription required) in a specific subnet or in the current local area network (if the subnet is not specified).

IPv4 uses a 32 bit number for the address of every networked device. The IP address is commonly represented as a sequence of four decimal numbers separated by dots, for example, 200.201.40.5 or 10.20.30.40. Each number represents one byte (8 bits) and can hence vary from 0 to 255.

The IP address is further broken down into the network ID and Host ID, which represent the network and the networked device (within the network) respectively. There have been a number of schemes used to allocate the 32 bit address space to networks. Historically, the network ID was just the first byte, meaning that there were only 256 possible networks. As the networks ran out, a new 'class'-based scheme was adopted, as shown in Figure 9.1.

Figure 9.1 New class based network scheme

The way this works is that the first bits are used to represent the 'class' of the address. Class A IP addresses start with 0 in the most significant bit of first byte - that is, they have the format 0XXXXXXXb ("b" indicates that the number is in binary format, and 'X' indicates that the field can be either one or zero). Class B, C, D and E addresses start with 10XXXXXXb, 110XXXXXb, 1110XXXXb, and 1111XXXXb respectively. Classes A, B, C are used for unicast, class D is used for multicast and class E is reserved.

The class of the address then determines how much of the rest of the number is used for the network ID and how much for the host. Class A addresses only use the first byte for the network address and the rest for host IDs, which implies that there are 127 class A networks (between 00000000b and 01111111b), each which have a very large number of computers in them. Class B addresses additionally use the second byte for the network ID while class C networks use both the second and third bytes. The result is that there are progressively more networks available to each of the address classes, each of which has fewer hosts (networked devices).

Subnet mask

The approach described is still a relatively inefficient method for allocating addresses to networks; it allows a lot more networks of varying sizes but still nowhere near enough. In order to extend the number of networks, a new partitioning scheme called Classless Inter-Domain Routing (CIDR) was created. In CIDR, a subnet mask is used to further subdivide networks, allowing us to arbitrarily specify which part of the IP address is the 'sub network ID' and which is the host ID.

The network mask is a 32 bit number that can be logically ORd with the IP address to get the network ID (it has 1 for all the left most bits that correspond to the network ID in the IP address). For example, consider the class A network address 10.0.0.0 with the network mask 255.255.255.0. This mask has 1 in the first 24 bits (three consecutive bytes with 255) creating the network ID 10.0.0 and leaving one additional byte for host ID. The CIDR notation for this mask is /24, an abbreviated expression to say the amount of bits set.

Two host IDs are reserved: the first address (all bits in host ID reset) is the network address. The last address (all bits in host ID set) is used for sending messages to all machines in this subnet, and is known as the broadcast address.

In summary:

Network: 10.0.0.0
Mask: 255.255.255.0 or /24 (CIDR notation)
IP addresses range: 10.0.0.1 to 10.0.0.254 (to be used in your machines)
Broadcast: 10.0.0.255

Let's examine a more complex example. Suppose you received the following network/mask from your Internet Service Provider (ISP):

Network: 200.201.145.128
Mask: 255.255.255.192 or /26 (CIDR notation)

The first two significant bits in the last byte of the mask are set (192d - decimal or 11000000b). This means that the first two significant bits of the last byte of the address (128 or 10b) are part of the network ID and only the last six bits are part of the host ID. Therefore, the network address is the value of the IP address with all the host ID bits reset to zero (000000b or 128d), and the broadcast address is the value with the last 6 bits set to 1 (10111111b or 191d).

In summary:

Mask: 255.255.255.192 or /26 (CIDR notation)
Network: 200.201.145.128
IP addresses range: 200.201.145.129 to 200.201.145.190 (to be used in your machines)
Broadcast: 200.201.145.191

Multicast addressing

IP multicast allows a networked application to communicate on a one-to-many basis, saving bandwidth and time.

Multicast IP addresses are special since they are not related to any specific network interface but instead to a group of machines. Networked devices that want to receive data on a multicast IP need to join to the group beforehand. Unfortunately, it is not possible understand multicast, and get the most out of the multicast sections later in this chapter, without first understanding how IP addresses are translated into physical addresses.

Each network interface has its own physical address called its MAC (Medium Access Control). The MAC is a 48 bit number that must be unique for each network interface within a subnetwork. This subnetwork may be understood as a physical network consisting of one or several network segments interconnected by layer one or layer two network devices such as repeaters and hubs (layer one) or bridges and switches (layer two). However, they may repeat if network segments are interconnected by IP routers (layer three). Typically, MAC address are represented as six hexadecimal numbers, with eight bits and separated by colons: 00:1E:68:BB:49:AB.

MACs are essential to deliver packages that come from the IP layer, since network interfaces only deal with physical addresses, not IPs (see Figure 9.2). So, in order to translate IP addresses (IP layer) into MAC addresses (data link layer), the ARP protocol is employed. Using ARP requests, tables for translating IP into MAC addresses are created in all networked nodes (called ARP cache), allowing translations and posterior delivery.

PythonOnSymbian tcp layers encaps addr.png

Figure 9.2 TCP layers

However, a question arises about multicast addressing: how to deliver a message if we have multiple destinations? In this case, a special MAC address not tied to any network interface is created using a common and fixed prefix (01:00:5E) and part of the original multicast IP. Moreover, all nodes that belong to that multicast group must program their network interface to receive messages addressed to that specific MAC address and not only to their own MAC address. Figure 9.3 shows how the multicast IP 224.202.20.30 is mapped to the MAC address 01:00:5E:4A:14:1E.

PythonOnSymbian multicast2mac.png

Figure 9.3 Multicast IP mapping

Special socket options are used to join multicast groups, creating special filters for corresponding MAC addresses in the network interface.

Ports

The IP address is used to locate a machine within a specific network, while the port is used to access the specific service. A single machine may run a number of IP based services (e.g. an FTP or web server), using a different port for each one.

The common analogy used to describe the relationship between IP addresses and ports is a company switchboard. You dial the company phone number to get the switchboard (which is analogous to the IP address), and then the extension number (Port) is used to get a specific person.

In a URL, the port is usually specified after the address, preceded by a colon:

http://www.example.com:8000

TCP and UDP

IP networking uses a transport protocol to define how the communication will be performed. TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are the most common transport methods:

  • TCP is a connection oriented protocol - it provides a reliable and ordered byte stream between two addresses (it's similar to making a telephone call to someone).
  • UDP uses a simpler "connection-less" model based on messages. UDP does not have any order or delivery guarantee (it's similar to sending several letters to someone).

TCP manages the reliable and ordered delivery of packets, making it the obvious protocol in many cases. UDP is a more lightweight protocol, and may be a good choice where reliable delivery is not an issue, or where reliability can be easily implemented in an application-specific protocol.

The Python Socket API

This section provides an overview of the socket API, showing how to set up mobile clients and servers and how to detect and connect to access points.

The discussion is illustrated with the aid of two examples:

  • the fortune client and server example shows a mobile client that receives random messages from a PC based server.
  • the 'homebrew' example shows a mobile Python client that waits for files sent from a PC based server.

Fortune client/server

Tip.pngTip: In this example, our server is running on a PC with an IP address of 10.0.0.100. You need to modify the client and server scripts to use the IP address of your local network - which can find out by starting a command prompt/terminal and typing ipconfig (Windows) or ifconfig (Linux)

The fortune server sends random messages from the Zen of Python to the fortune client over an ad-hoc unicast network. The example assumes that you have a Symbian device and a PC connected to a WiFi router, as shown in Figure 9.4. The client application runs on the device and the server runs on the PC. The server could also run on a device, but this way you get to see how similar Python network programming is on the PC and on Symbian.

Figure 9.4 Fortune client/server example hardware configuration

The fortune client code is as follows (note that some details, including exception handling and network configuration are omitted for the sake of clarity).

The script first imports the socket module in order to access all the socket networking functions. It then defines the address of the server and the port where the service is running (these must be published by the server).

The socket() function creates a network file descriptor, very similar to the one received when a file is opened. The function sets the address family and the transport protocol to IPv4 and TCP using socket options. Common address families are AF_INET (IPv4 addresses) and AF_INET6 (IPv6 addresses). TCP can be selected with SOCK_STREAM and UDP with SOCK_DGRAM. All of these constants are located inside the socket module.

# Fortune client
from socket import *
from appuifw import note
 
HOST = "10.0.0.100"
PORT = 54321
 
s = socket(AF_INET,SOCK_STREAM)
s.connect((HOST,PORT))
 
fortune = u""
while True:
data = s.recv(1024)
if not data:
break
fortune += data
 
s.close()
 
note(fortune,"info")

After socket creation we call connect() to initiate the connection to the fortune server, specifying a tuple with IP (or host domain name such as symbian.org) and port. Since the transport protocol was set when we created the socket, all elements for accessing the server are now prepared. At this point an access point selection dialog is presented to the client, and connection will take place once an access point is selected.

Tip.pngTip: If you have a firewall running on the PC side, it is necessary to open the incoming port 54321. Otherwise your mobile client will not be able to connect to the server.

The recv() function is called within a loop to wait for a message. The bytes received each time recv() returns are added to the fortune buffer. When the server has finished sending the message it terminates the connection; the client recv() returns with an empty string, which causes the loop to terminate. The client then closes the connection to its socket and displays the received message to the user (closing the socket is not strictly necessary, because it would otherwise be automatically closed when garbage-collected).

Note.pngNote: The recv() function specifies at least one argument - the maximum number of bytes that may be received when the function returns. A common misunderstanding is that each recv() will receive a complete message, or that it will receive the amount of bytes specified in the call. Network and operating system conditions (delay, buffers, throughput and so on) may change the amount of bytes received at each recv() call. Even though the streaming is guaranteed by TCP, the programmer must still check for protocol integrity.

The server code shown as follows is relatively straightforward - it creates a TCP/IP socket, binds to the address and port and waits (listens) for connections. When a connection is received it selects a random message, sends it, then closes the connection.

# Fortune server
from socket import *
from random import choice
 
# Fortune database from The Zen of Python, by Tim Peters (import this)
PHRASES =[ u"Beautiful is better than ugly.",
u"Explicit is better than implicit.",
u"Simple is better than complex.",
u"Complex is better than complicated.",
u"Flat is better than nested.",
u"Sparse is better than dense.",
u"Readability counts.",
u"Special cases aren't special enough to break the rules.",
u"Although practicality beats purity.",
u"Errors should never pass silently.",
u"Unless explicitly silenced.",
u"In the face of ambiguity, refuse the temptation to guess.",
u"There should be one-- and preferably only one --obvious way to do it.",
u"Although that way may not be obvious at first unless you're Dutch.",
u"Now is better than never.",
u"Although never is often better than *right* now.",
u"If the implementation is hard to explain, it's a bad idea.",
u"If the implementation is easy to explain, it may be a good idea.",
u"Namespaces are one honking great idea -- let's do more of those!" ]
 
HOST = "10.0.0.100"
PORT = 54321
 
s = socket(AF_INET,SOCK_STREAM)
s.bind((HOST,PORT))
s.listen(5)
 
while True:
(cs,addr) = s.accept()
fortune = choice(PHRASES)
cs.sendall(fortune)
cs.close()

The server script is run on the PC and the client script in the Python shell on the device. Screen shots of messages being received on the mobile device are shown in Figure 9.5 as follows:

Figure 9.5 Screen shots of messages being received on the mobile device

Access point selection

On a mobile device there may be a number of potential connections to the Internet: via the phone network, WiFi, or a tethered connection to a PC. By default, Python will pop up a dialog when a socket calls connect() to allow users to choose which connection they want to use.

Python also provides a number of functions to allow applications to query the set of available access points and set a default connection. This allows access points to be stored and reused (reducing the number of prompts received by the user):

  • access_points(): Returns a list of all the available access point ids and their names
  • set_default_access_point(ap_name): Sets the default access point to be used when the socket is opened and starts the connection.

These methods make it easy to create a custom access point selection dialog. The code and a screenshot of such a dialog are shown as follows and in Figure 9.6 respectively.

# Selecting access point
from appuifw import *
import socket
 
def sel_access_point():
""" Select the default access point.
Return True if the selection was done or False if not
"""

aps = socket.access_points()
if not aps:
note(u"No access points available","error")
return False
 
ap_labels = map(lambda x: x['name'], aps)
item = popup_menu(ap_labels,u"Access points:")
if item is None:
return False
 
socket.set_default_access_point(aps[item]["name"])
 
return True
 
if sel_access_point():
note(u"Connect!","info")
Figure 9.6 Custom access point selection dialog

Once you have connected to your destination it is possible to get your own IP address and port by calling the socket method getsockname():

>>> s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
>>> s.connect(("10.0.0.100",54321))
>>> s.getsockname()
('10.0.0.102', 54838)

If you need to know this IP address prior to connecting to your destination, you can use the btsocket module. This module provides much the same functionality as socket but adds support for sockets over Bluetooth connections. For most operations the modules can be used interchangeably. Unfortunately btsocket cannot be used as a direct replacement for socket in all cases because it does not expose all of the socket options (this causes problems when creating a multicast server). In addition, it does not support the socket time-outs (settimeout(), discussed later).

If you want to use btsocket, the following code can be used to select the access point (this function will be used in other examples, copy it when necessary).

# Selecting access point
from appuifw import *
import btsocket
 
def sel_access_point_bt():
""" Select the default access point.
Return the acess point if the selection was done
or None if not
"""

aps = btsocket.access_points()
if not aps:
note(u"No access points available","error")
return None
 
ap_labels = map(lambda x: x['name'], aps)
item = popup_menu(ap_labels,u"Access points:")
if item is None:
return None
 
apo = btsocket.access_point(aps[item]['iapid'])
btsocket.set_default_access_point(apo)
 
return apo
 
apo = sel_access_point_bt()
if apo:
apo.start()
note(u"Connect! IP "+apo.ip(),"info")

This function returns an access point object, which has three interesting methods:

>>> dir(apo)
['ip', 'start', 'stop']

The start() function can be used to activate the access point, and you can interrupt it with apo.stop(). If you are using DHCP, the IP address will be negotiated in this phase and may be retrieved using the apo.ip() function. These functions are used when sending/receiving datagrams (UDP, multicast or broadcast messaging), and with these types of messages there is no connection phase so you manage the access point directly.

When the access point is running, the phone will show the connection icon on the phone home screen as shown in Figure 9.7.

Figure 9.7 Phone in which a connection is active

Take care when mixing btsocket and socket:

  • If you want to fix the access point you will need to select the access points for both modules.
  • Some Python standard modules will import socket instead of btsocket, resulting in unexpected behaviour. In this case, we recommend that you add the following lines to the beginning of your first script:
import sys
try:
# Try to import btsocket as socket
sys.modules['socket'] = __import__('btsocket')
except ImportError:
pass

This ensures that any further script that tries to import socket will import btsocket instead.

Protocols and TCP servers

In this section, we create a single-thread server to receive files over a WiFi network. Initially, we define a PC client script to send the files, and we then demonstrate a client that can be hosted on a mobile device.

The client and server rely on a simple communication protocol (a communication protocol is a set of agreed rules for data representation, signaling, authentication and error detection that can be used to share information between two end points). The protocol doesn't need to be complicated or based on any particular standard, it just needs to be understood by both the server and the client.

Our 'homebrew file transfer protocol' has three fields, separated by \n:

  • A string representing the file name (maximum of 32 characters)
  • The size of the file, specified to be big endian (most significant byte first in memory).
    • Endian-ness is important when exchanging data between machines with different architectures to ensure it can be unpacked properly (PCs are little endian while the processors on mobile devices are usually big endian).
    • The size can be used for providing progress notification or confirming that there is enough space for the received file (we don't actually use it in this example).
  • File contents

The protocol is shown in Figure 9.8:

Figure 9.8 File format of a "homebrew" file message

TCP servers use the following sequence of calls to set up the connection with a client:

  1. socket(): Creates a socket object to provide the sockets API
  2. bind(): Binds the socket (server) to a specific IP address and port where clients will expect to find the service (binding tells the operating system that events on the specified IP address and port should be directed to the application). For multi-homed machines (machines with several IP addresses), we can specify an IP address of 0.0.0.0 to tell the operating system that we want to bind the server to all available local IP addresses (but the same port).
  3. listen(): Reserves resources in the device, telling it how many simultaneous connection requests it must be able to handle on the current socket, after which new connection requests must be dropped. Once a connection is established its reserved resources are released, allowing further connection requests. The number should be the same as the likely number of simultaneous connection requests, noting of course that larger numbers will consume more resources. The number doesn't affect how many connections a server may have active, just how many may be established at the same time.
  4. accept(): Blocks the thread waiting on incoming connections. At this point everything is defined and the server is up and running. When a connection arrives, the program continues its normal execution flow just after the accept() call. Note that it is possible to make accept() non-blocking with additional socket options.

accept() returns two important parameters:

  • a socket for describing the incoming connection.
  • client IP address and port, as a tuple.

From this point, all further communication with the client must be done using the returned socket. Data may be exchanged using recv() and send(). Multi-threaded servers will create a new thread and pass the socket to it, then call accept() again to wait for more clients.

The server code is as follows. The server is a Python application that creates a custom access point selection dialog (as discussed in the preceding section), and an 'About' dialog. The code to set up the connection follows the pattern described.

# File upload server
import sys
try:
# Try to import btsocket as socket
sys.modules['socket'] = __import__('btsocket')
except ImportError:
pass
import socket
from appuifw import *
import os
import e32
import struct
 
class FileUpload(object):
""" File upload server class
"""

def __init__(self):
self.lock = e32.Ao_lock()
self.dir = "e:\\file_upload"
if not os.path.isdir(self.dir):
os.makedirs(self.dir)
self.apo = None
self.port = 54321
self.new_line = u"\u2029"
app.title = u"File upload"
app.screen = "normal"
app.menu = [(u"About", self.about)]
self.body = Text()
app.body = self.body
 
def recv_file(self,cs,addr):
""" Given a client socket (cs), receive a new file
and save it at self.dir
"""

data = ""
name = ""
size = 0
# waiting for file name
while True:
n = data.find("\n")
if n >= 0:
name = data[:n]
data = data[n+1:]
break
buf = cs.recv(1024)
data += buf
 
# waiting for file size (may be useful for limits checking)
while True:
n = data.find("\n")
if n >= 0:
# unpack one long (L) using big endian (>) endianness
size = struct.unpack(">L",data[:n])[0]
data = data[n+1:]
break
buf = cs.recv(1024)
data += buf
 
self.body.add(u"Uploading %s (%d bytes)" % (name,size) + self.new_line)
# waiting for file contents
fname = os.path.join(self.dir,name)
f = open(fname,"wb")
while True:
f.write(data)
data = cs.recv(1024)
if not data:
break
self.body.add(u"Finished." + self.new_line)
cs.close()
f.close()
 
def server(self,ip,port):
""" Starts a mono thread server at ip, port
"""

s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.bind((ip,port))
s.listen(1)
 
while True:
(cs,addr) = s.accept()
self.body.add(u"Connect to %s:%d" % (addr[0],addr[1]) + self.new_line)
self.recv_file(cs,addr)
 
def sel_access_point(self):
""" Select and set the default access point.
Return the access point object if the selection was done or None if not
"""

aps = socket.access_points()
if not aps:
note(u"No access points available","error")
return None
 
ap_labels = map(lambda x: x['name'], aps)
item = popup_menu(ap_labels,u"Access points:")
if item is None:
return None
 
apo = socket.access_point(aps[item]['iapid'])
socket.set_default_access_point(apo)
 
return apo
 
def about(self):
note(u"File upload for PySymbian","info")
 
def run(self):
self.apo = self.sel_access_point()
if self.apo:
self.apo.start()
self.body.add(u"Starting server." + self.new_line)
self.body.add(u"IP = %s" % self.apo.ip() + self.new_line)
self.body.add(u"Port = %d" % self.port + self.new_line)
self.body.add(u"Repository = %s" % (self.dir) + self.new_line)
self.server(self.apo.ip(),self.port)
self.lock.wait()
app.set_tabs( [], None )
app.menu = []
app.body = None
app.set_exit()
 
if __name__ == "__main__":
app = FileUpload()
app.run()

When compared to the fortune client-server example, the code to process a 'homebrew' message is relatively complicated. This is because the data does not arrive in one go and we need to parse it as it arrives in order to get the filename and size. As discussed in the section "File-like sockets" the makefile() function can make processing easier by reading a specified amount of data or a whole 'line'.

Finally, the small PC script shown as follows may be used to send the file:

# File upload client
import struct
import sys
import os
import socket
 
if len(sys.argv) < 4:
print "%s server_addr server_port file_to_upload" % sys.argv[0]
sys.exit(1)
 
ip = sys.argv[1]
port = int(sys.argv[2])
full_name = sys.argv[3]
base_name = os.path.basename(full_name)
size = os.path.getsize(sys.argv[3])
 
print "Sending %s to %s:%d ..." % (base_name,ip,port)
 
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect((ip,port))
f = open(full_name,"rb")
header = "%s\n%s\n" % (base_name,struct.pack(">L",size))
s.sendall(header)
while True:
data = f.read(1024)
if not data:
break
s.sendall(data)
 
s.close()
f.close()

Run it from command line, specifying the server IP address, port, and the file to be sent:

script_name.py server_ip server_port file_to_send

Figures 9.9 and 9.10 show these programs running. The command ping was used to test the ad-hoc connection between PC (10.0.0.100) and mobile phone (10.0.0.103), before transferring files.

Figure 9.9 PC client sending file
Figure 9.10 Mobile server receiving file

Finally, the following code is for a client that you can run on the device to send the files. It is similar to the PC script but the main difference is that it adds a UI wrapper and a class for selecting files (FileSel).

Take a look in the source code and try to understand how it works. Some socket calls are surrounded by try... catch statements (exceptions) and we will talk about them later.

# File upload client
import sys
try:
# Try to import btsocket as socket
sys.modules['socket'] = __import__('btsocket')
except ImportError:
pass
import socket
from appuifw import *
import os
import e32
import struct
import time
import re
 
class FileSel(object):
"""
Open a selection file dialog. Returns the file selected or None.
Initial path and regular expression for filtering file list may be provided.
 
Examples:
sel = FileSel().run()
if sel is not None:
...
 
sel = FileSel(mask = r"(.*\.jpeg|.*\.jpg|.*\.png|.*\.gif)").run()
if sel is not None:
...
 
"""

def __init__(self,init_dir = "", mask = ".*"):
self.cur_dir = unicode(init_dir)
if not os.path.exists(self.cur_dir):
self.cur_dir = ""
self.mask = mask
self.fill_items()
 
def fill_items(self):
if self.cur_dir == u"":
self.items = [ unicode(d + "\\") for d in e32.drive_list() ]
else:
entries = [ e.decode('utf-8')
for e in os.listdir( self.cur_dir.encode('utf-8') ) ]
d = self.cur_dir
dirs = [ e.upper() for e in entries
if os.path.isdir(os.path.join(d,e).encode('utf-8')) ]
 
files = [ e.lower() for e in entries
if os.path.isfile(os.path.join(d,e).encode('utf-8')) ]
 
files = [ f for f in files
if re.match(self.mask,f) ]
dirs.sort()
files.sort()
dirs.insert( 0, u".." )
self.items = dirs + files
 
def run(self):
while True:
item = selection_list(self.items, search_field=1)
if item is None:
return None
f = self.items[item]
d = os.path.abspath( os.path.join(self.cur_dir,f) )
if os.path.isdir( d.encode('utf-8') ):
if f == u".." and len(self.cur_dir) == 3:
self.cur_dir = u""
else:
self.cur_dir = d
self.fill_items()
elif os.path.isfile( d.encode('utf-8') ):
return d
 
class TxFile(object):
""" TxFile client class
"""

def __init__(self):
self.lock = e32.Ao_lock()
self.apo = None
self.dir = ""
self.port = 54321
self.ip = ""
self.new_line = u"\u2029"
app.title = u"TX File"
app.screen = "normal"
app.menu = [(u"Send file", self.send_file),
(u"Set AP", self.set_ap),
(u"Exit", self.close_app)]
self.body = Text()
app.body = self.body
 
def close_app(self):
self.lock.signal()
 
def set_ap(self):
""" Try to set an access point, return True or False to indicate the success.
If True, sets self.apo to choosen access point.
"""

apo = self.sel_access_point()
if apo:
self.apo = apo
return True
else:
return False
 
def send_file(self):
""" Send a file to server
"""

# at leat one access point is necessary
if not self.apo:
if not self.set_ap():
return
 
# use our own IP as initial guess
self.apo.start()
if not self.ip:
self.ip = self.apo.ip()
 
# get server address
ip = query(u"Server addr", "text", unicode(self.ip))
if ip is None:
return
self.ip = ip
 
# get filename
full_name = FileSel(init_dir=self.dir).run()
if full_name is None:
return
 
# transmitt file
full_name = full_name.encode('utf-8')
self.dir = os.path.dirname(full_name)
base_name = os.path.basename(full_name)
size = os.path.getsize(full_name)
 
 
self.body.add(u"Connecting to %s:%d ..." % (self.ip,self.port) + self.new_line)
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
try:
s.connect((self.ip,self.port))
except socket.error, (val,msg):
self.body.add(u"Error %d: %s" % (val,msg) + self.new_line)
return
 
self.body.add(u"Sending %s (%d bytes)" % (base_name,size) + self.new_line)
f = open(full_name,"rb")
header = "%s\n" % (base_name) + struct.pack(">L",size) + "\n"
s.sendall(header)
n = 0
ta = time.time()
while True:
data = f.read(1024)
if not data:
break
s.sendall(data)
n += len(data)
if n % 100 == 0: # a mark at each 100k
self.body.add(u".")
s.close()
f.close()
tb = time.time()
self.body.add(self.new_line + u"Finished (%0.2f kbytes/s)." % ((n/1024.0)/(tb-ta)) + self.new_line)
 
def sel_access_point(self):
""" Select and set the default access point.
Return the access point object if the selection was done or None if not
"""

aps = socket.access_points()
if not aps:
note(u"No access points available","error")
return None
 
ap_labels = map(lambda x: x['name'], aps)
item = popup_menu(ap_labels,u"Access points:")
if item is None:
return None
 
apo = socket.access_point(aps[item]['iapid'])
socket.set_default_access_point(apo)
 
return apo
 
def run(self):
self.lock.wait()
app.set_exit()
 
if __name__ == "__main__":
app = TxFile()
app.run()

Exception handling and debugging

The last section raised the issue of exception handling: the socket API on Python on Symbian makes extensive usage of exceptions to indicate errors like connection timeouts, unexpected closed connections and so on.

Tip.pngTip: You can use the settimeout() socket method to set the time out value for any socket operation, including the connection time. Note that this method is available only in the socket() module and not in btsocket.

For instance, the following code fragment shows how to use exception handling when connecting to a server.

# btsocket can be used as well
import socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
try:
s.connect(('nokia.com',2222))
except:
print "Can´t connect"

Since port 2222 does not have a server listening on it, an exception will be raised and the script will print Can't connect.

The code catches every exception, but doesn't tell us very much about the problem. We can obtain more information by using the socket-specific exception: socket.error. In the following example, if there is an exception we get the more useful message ERROR: timed out.

# btsocket can be used as well
import socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
try:
s.connect(('nokia.com',2222))
except socket.error, e:
print "ERROR: %s" % e

Another useful exception is socket.gaierror, related to address resolution problems. In the next example an invalid address is used ('nokiacom'), generating an exception. Note how we use several except statements.

# btsocket can be used as well
import socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
try:
s.connect(('nokiacom',2222))
except socket.gaierror, e:
print "ADDR ERROR: %s" % e
except socket.error, e:
print "ERROR: %s" % e
except:
print "Can't connect"

The output is as shown, with the most specific exception displayed first:

ADDR ERROR: (11001, 'getaddrinfo failed')

Following the 'Zen of Python', 'explicit is better than implicit' and 'errors should never pass silently', it is better to handle exceptions individually. However, the 'lazy approach' (first example) may be enough in many situations.

Exception handling can be used to help your program deal with unexpected and environmental errors. To deal with programmatic errors you're going to need the techniques discussed in Chapter 17, which include guidance on how to use the emulator and how to use file logging.

Advanced Socket API

Multicast

The previous examples demonstrated unicast networking with TCP. Multicast and broadcast networking uses UDP, because it is inherently message-based rather than data-stream based. This section explains how to send multicast messages in your applications and, as a bonus, provides an overview of UDP networking.

The following Python code sends ten 'hello world' messages every two seconds to the multicast address 224.202.20.30. The screenshot in Figure 9.11 shows the multicast frame, addressed to MAC 01:00:5e:4a:14:1e:

# Sending multicast from mobile
import e32
import btsocket
from appuifw import note, popup_menu
 
GROUP = "224.202.20.30"
PORT = 54321
 
apo = sel_access_point_bt()
if apo is not None:
apo.start()
sock = btsocket.socket(btsocket.AF_INET, btsocket.SOCK_DGRAM)
for i in range(10):
n = sock.sendto('hello world',(GROUP,PORT))
print "Message sent (%d bytes)" % n
e32.ao_sleep(2)

It is possible to use the socket module as well, by changing the access point selection routine:

# Sending multicast from mobile
import e32
import socket
from appuifw import note, popup_menu
 
GROUP = "224.202.20.30"
PORT = 54321
 
if sel_access_point():
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for i in range(10):
n = sock.sendto('hello world',(GROUP,PORT))
print "Message sent (%d bytes)" % n
e32.ao_sleep(2)

The code is fairly straightforward:

  • Select an access point, set it as the default access point. For those running btsocket, use the apo object to start the access point. UDP is a connection-less protocol, so we need to set the access point directly.
  • Create a UDP socket (the transmission control protocol is set with SOCK_DGRAM instead of SOCK_STREAM).
  • Call sendto() in a loop to send the "hello world" message to the specified group and port.

sendto() allows us to specify the message destination in the format (address, port). We use this instead of the more familiar send() function because there is no mandatory connection phase in UDP networking. Note that you can use the send()/recv() primitives instead non-connected versions sendto()/recvfrom() if you want, but you'll first need to connect the UDP socket.

Figure 9.11 Wireshark screenshot showing multicast frame

To receive multicast messages, we first create a socket and bind it to the interface. We then join the desired group, as discussed in the Multicast addressing section at the beginning of this chapter. This can be done using two calls to setsockopt() function. The first call enables multicast reception for the interface and the second tells the network card which IP we are waiting for.

After joining the group, the last step is to call recvfrom() to wait for a message. Again, recvfrom() is similar to recv() but it additionally returns the address of the incoming package.

Note.pngNote: At time of writing Python on Symbian v2.0 allows applications to send but not receive multicast messages using btsocket or even socket. While a group of options SOL_IP is not present in the btsocket module, socket returns a Error 17: File exists when you call setsockopt using multicast parameters.

As Python on Symbian does not enable multicast messages to be received, the following code is run on the PC:

# PC multicast server
import time
import sys
import socket
 
GROUP = "224.202.20.30"
PORT = 54321
ITF_ADDR = '10.0.0.101'
 
def join_grp(sock,grp,itf):
# enabling multicasting option for interface itf
sock.setsockopt(socket.SOL_IP,
socket.IP_MULTICAST_IF,
socket.inet_aton(itf))
# joining to multicast group grp in interface itf
sock.setsockopt(socket.SOL_IP,
socket.IP_ADD_MEMBERSHIP,
socket.inet_aton(grp)+socket.inet_aton(itf))
 
def leave_grp(sock,grp,ITF_ADDR):
# removing
sock.setsockopt(socket.SOL_IP,
socket.IP_DROP_MEMBERSHIP,
socket.inet_aton(grp)+socket.inet_aton(ITF_ADDR))
 
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind((ITF_ADDR,PORT))
join_grp(sock,GROUP,ITF_ADDR)
 
for n in range(10):
(data,addr) = sock.recvfrom(1500)
print "Received ",data,"from",addr
 
leave_grp(sock,GROUP,ITF_ADDR)

The code may at first appear a little opaque. It may help to know that the inet_aton() method converts an IPv4 address to 32-bit packed binary format as a string of four characters (see Python docs). Also, that setsockoption() has three parameters:

  • level: used to select a group of options, like SOL_SOCKET for socket options and SOL_IP for options related to IP layer.
  • option name: the current value of the option
  • option value: the new value of the option
Figure 9.12 Received messages displayed on PC python console

Broadcast

Broadcast messaging allows programs to send UDP messages to all machines in a subnet by specifying the destination as a specific broadcast address. All machines in the subnet will receive these broadcast messages without need to register.

There are four types of broadcast (the first two are the most common):

  • Limited broadcast: A special broadcast where the destination address is 255.255.255.255. It is limited because only machines in the local subnet will receive this message - messages are never forwarded to other networks.
  • Subnet directed broadcast: The subnet broadcast IP address is used to specify the target subnet. Routers may forward this message but they need to know the subnet mask. For instance, the broadcast address for network 172.16.10.0 with mask 255.255.255.0 is 172.16.10.255.
  • Net-directed broadcast: The broadcast address is calculated using the default mask for class A, B or C addresses. Class A will have a broadcast address like <netid>.255.255.255, for instance. Routers will forward these messages, but it is possible to disable them.
  • All subnets directed broadcast: Quite similar to net directed broadcast, but in this approach all subnets are grouped. For example, 172.16.255.255 is the broadcast address for all subnets-directed to network 172.16.10.0.

In the data link layer, broadcast messages use the special MAC address FF:FF:FF:FF:FF:FF regardless the type of broadcast IP address used. The screenshot shown in Figure 9.13 shows this MAC used for the subnet broadcast address 10.0.0.255 for network 10.0.0.0/24.

File:PythonOnSymbian broadcast pc.png
Figure 9.13 Wireshark showing frame for broadcast message

The following code sends broadcast messages. We set the socket option SO_BROADCAST to true in SOL_SOCKET. Note that the code will also run on the PC if you replace e32.ao_sleep(2) by time.sleep(2).

Note.pngNote: This example will work only for the socket module since the SOL_SOCKET group of options is not present in the btsocket module.)

# Sending broadcast messages
import socket
import e32
 
BCIP = "255.255.255.255"
PORT = 54321
 
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET,socket.SO_BROADCAST, True)
for i in range(10):
n = sock.sendto('hello world',(BCIP,PORT))
print "Message sent (%d bytes)" % n
e32.ao_sleep(2)

The following example shows a code snippet for receiving broadcast messages.

# Receiving broadcast messages
import socket
from appuifw import popup_menu
 
PORT = 54321
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind(('0.0.0.0',PORT))
 
if sel_access_point():
for i in range(10):
(data,addr) = sock.recvfrom(1500)
print "Received ",data,"from",addr

It is possible to use the socket module as well, changing the access point selection routine:

# Receiving broadcast messages
import btsocket
from appuifw import popup_menu
 
PORT = 54321
sock = btsocket.socket(btsocket.AF_INET,btsocket.SOCK_DGRAM)
sock.bind(('0.0.0.0',PORT))
 
apo = sel_access_point_bt()
if apo:
apo.start()
for i in range(10):
(data,addr) = sock.recvfrom(1500)
print "Received ",data,"from",addr

File-like sockets

TCP sockets return an arbitrary amount of data with every recv() call. As a result, we end up either buffering the whole message in memory before parsing it (resulting in less memory efficient code) or parsing it on-the-fly (resulting in more complicated code).

Python's socket API provides methods that make processing streamed data a lot easier. The makefile() function transforms a standard socket object in an object with file-like behavior. With this file-like socket you can read() a fixed amount of bytes or readline() to receive a complete line. Being able to read a known size of data significantly reduces the complexity of the data processing code.

Note.pngNote: While frequently used in Python applications, this object not is common in other socket APIs

The first example is a line-oriented client-server. The client code (displayed first) sends five lines to the server through the file-like object. The newline character is appended to each line of text - this is what is used to detect lines of data by the receiver. A call to flush() is added before calling time.sleep() to flush any pending bytes.

# File-like sockets - client
# btsocket can be used as well
import socket
import e32
 
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect(('127.0.0.1',54321))
fsock = sock.makefile()
 
for i in range(5):
msg = u"Sending line %d ...\n" % i
fsock.write(msg)
print msg
# call flush to send pending bytes
fsock.flush()
# slow down for better comprehension
e32.ao_sleep(1)
 
sock.close()

The server is simple:

# File-like sockets - server
# btsocket can be used as well
import socket
 
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.bind(('127.0.0.1',54321))
s.listen(1)
 
while True:
sock,addr = s.accept()
fsock = sock.makefile()
print "New connection from",addr
while True:
line = fsock.readline()
if not line:
print "Connection closed"
break
print "=> ",line

File-like sockets are even more useful when we need to decode headers or other protocol information. Suppose we have created a protocol in which we send the version and size as two four-byte integers (big endian format) followed by the data. The following client and server code shows how makefile makes this easy:

# File-like sockets - proto client
# btsocket can be used as well
import socket
import struct
import random
 
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect(('127.0.0.1',54321))
 
fsock = sock.makefile()
# creating a random header
version = random.randint(0,5)
size = random.randint(0,10)
header = struct.pack('>LL',version,size)
print "Version:",version
print "Size:",size
fsock.write(header)
data = 'X'*size
print "Data:",data
fsock.write(data)
fsock.flush()
sock.close()

Note.pngNote: The struct module is useful when handling data stored in files or from network connections.

The server is similar:

# File-like sockets - proto server
# btsocket can be used as well
import socket
import struct
 
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.bind(('127.0.0.1',54321))
s.listen(1)
while True:
sock,addr = s.accept()
fsock = sock.makefile()
print "New connection from",addr
# decoding header
header = fsock.read(8)
(version,size) = struct.unpack('>LL',header)
print "Version:",version
print "Size:",size
data = fsock.read(size)
print "Data:",data
sock.close()

Further reading

The following books use C as a programming language to explain the socket API. The behaviour of Python sockets is similar because the C socket API has become a de facto standard across most programming languages.

  • Beej's Guide to Network Programming
    • Concise and straight to the point, this guide remains a great reference.
  • TCP/IP Illustrated, Volume 1, 1994, by W. Richard Stevens
    • Despite being quite old, this is still a fantastic book. It covers in details several important protocols, including: TCP, UDP, IP, ICMP and others.
  • TCP/IP Illustrated, Volume 2
    • This provides implementation information and source code for BSD 4.4, the starting point for many network stack implementations.
  • UNIX Network Programming, Third edition, 2004, by W. Richard Stevens, Bill Fenner and Andrew M.Rudoff.
    • This book was originally written by Stevens but, after he passed away in 1999, it was released in a new edition. The book discusses both the socket API and more recent protocols, like IPv6 and SCTP. It is a book for programmers and a great reference.

Several books have been written specifically for Python developers:

Source Code

All examples shown in this chapter are available for download from File:PythonOnSymbianBookExampleCode Ch09BasicNetProg.zip.

Summary

This chapter shows how to program networked applications on Symbian devices using Python's socket APIs. Covering both the fundamental principles of IP networking and also aspects of sockets programming that are specific to mobile devices, the text is complemented by simple but illustrative examples of Python network clients and servers. The topic also covers more advanced IP networking topics, including multicast and broadcast networks, and debugging networking code.

Chapter 14, which covers advanced network programming, extends the concepts in this chapter to discuss higher level networking protocols (e.g. HTTP, XML-RPC), and to cover the essentials of multi-threaded network programming.

Licence icon cc-by-sa 3.0-88x31.png© 2010 Symbian Foundation Limited. Portions copyright Bogdan Galiceanu, Hamish Willee, Marcelo Barros de Almeida, Mike Jipping, Pankaj Nathani and others in wiki document history list. This document is licensed under the Creative Commons Attribution-Share Alike 2.0 license. See http://creativecommons.org/licenses/by-sa/2.0/legalcode for the full terms of the license.
Note that this content was originally hosted on the Symbian Foundation developer wiki.

This page was last modified on 8 May 2013, at 08:53.
110 page views in the last 30 days.
×