×
Namespaces

Variants
Actions

Symbian Platform Support for Writeable Static Data in DLLs

From Nokia Developer Wiki
Jump to: navigation, search
Article Metadata
Article
Created: hamishwillee (30 Sep 2007)
Last edited: hamishwillee (25 Aug 2011)

The Symbian platform supports global writeable static data (WSD) in both kernel and user side DLLs and EXEs, and on all versions and handsets. However the support for WSD in user-side DLLs has potentially significant costs in terms of RAM and chunk resource usage, and creates tight constraints on what can be done with the Symbian emulator.

Application/ISV developers will often find the costs of WSD in DLLs acceptable, particularly when porting code from other less resource-constrained operating systems. This is particularly true when the DLL to be ported includes significant amounts of WSD.

Device Creation Developers will find the trade offs are different; the costs and constraints mean that using global data in DLLs is unlikely to be justifiable.

This document explains what WSD is and how you enable it for your DLLs, the costs and caveats, and some of the alternatives. The Frequently Asked Questions includes some explanations of our implementation.

Note.pngNote: This document discusses only the WSD support on the Symbian platform; it supersedes SupportForWriteableStaticDataInDLLsv2.2.pdf that additionally covers the platform precursor (Symbian OS).

Contents

What Is Global Writeable Static Data?

Global writeable static data (WSD) is any per-process variable which exists for the lifetime of the process. In practice, this means any globally scoped data - data that is declared outside of a function, struct, or class, and function scoped static variables.

TBufC<20> fileName; //WSD
void SetFileName()
{
static TInt iCount; //WSD
...
}

Platform code written in Symbian C++ code rarely uses WSD. However code ported from other operating systems may contain large amounts of static data. For example, code written in the C programming language often makes use of WSD as the “glue” between C function calls.

A common misunderstanding is that const global variables are read-only data rather than global writeable static data [1]. This is true for const objects with trivial constructors. However if a const class has a non-trivial constructor, the const object will require a real variable and must be stored as WSD. For example:

const TInt myVariable=;        //OK – truly const
const TPtrC KSomeConstPtr=...; //NOT OK – non trivial constructor
const TRgb KSomeConstCol=...; //NOT OK – non trivial constructor

Enabling Global Writeable Static Data

In order to enable global writeable static data, simply add the EPOCALLOWDLLDATA (case insensitive) to your MMP file:

TARGET                  my.dll
TARGETTYPE dll
EPOCALLOWDLLDATA
...

WSD: Costs and Caveats

The Emulator Only Allows a DLL With WSD to Load Into a Single Process

Note.pngNote: This limitation applies to Symbian^2 and earlier. From Symbian^3 WSD is transparently supported in the Emulator

On Symbian^2 and earlier versions, the platform emulator only allows a DLL with WSD to be loaded into a single process (the reasons for this limitation are discussed in #Why Does the EKA2 Emulator Only Allow A DLL With WSD To Load Into A Single Process?).

This is not a problem for most application developers, where a DLL is likely to only be loaded into the (single) application executable. However if you have a share DLL with WSD, then the second process that attempts to load it in the emulator will fail with KErrNotSupported.

Symbian^1 provides non-transparent emulator-specific support for process-wide WSD through the “EWSD” library. This mechanism allows developers to load a DLL with WSD into multiple processes on the emulator, at the cost of potentially significant additional development effort. A brief overview of the EWSD support, and the costs, is provided in #Emulator Support For WSD.

For reasons described in #What Happens If 'epocallowdlldata' Isn’t Declared For A DLL With WSD?, the emulator will allow WSD in DLLs if EPOCALLOWDLLDATA is not declared in the MMP file. However the data will be truly global – there will be one copy for the entire emulator rather than one copy for each emulated process. The only restriction is that if the data’s initialisers call any Symbian OS kernel functions (i.e. executive calls) the emulator will fault.

RAM Usage for WSD Data Chunk

When a process loads its first DLL containing WSD it creates a single chunk to store the data. The data for subsequent WSD-enabled DLLs is loaded into the same chunk.

The data chunk will occupy at least one 4K RAM page (the smallest possible RAM allocation), irrespective of how much static data is required. Any memory not used for WSD will be wasted. Since the memory is per-process, the memory wastage on the machine is:

(4Kb – WSD Bytes) × number-client-processes

It is very easy for a developer to add a few words of WSD to their DLL thinking that’s all the memory that they are using. However the cost is actually 4K for every process if a DLL with WSD has not already been loaded into the process. If for example the DLL is used by 4 processes, that’s potentially an “invisible” cost of 16K.

For an ISV application developer porting/writing DLLs with a single client process the (approximately) 4Kb wasted memory may be justifiable.

For Device Creators, where much of the code written is shared DLLs, using WSD has proportionally greater costs. The largest possible amount of wastage is 400Kb - calculated by assuming that there are 100 processes running on the device, each loading a single DLL with 1 byte of WSD.

Chunks Are a Finite Resource on ARM v5

Every process loading WSD enabled DLLs uses a chunk to hold the data. On ARM v6, there is no chunk limit, and this is not an issue.

However on ARM v5, the Symbian platform has a hard coded limit of 16 chunks-per-process; a limit that is required to ensure real-time behaviour. Programs using more than 16 or more chunks need to be aware that one will already have been used for WSD.

ARM v4 & v5 architecture specific costs and limitations

There are other significant costs that apply only to DLLs that link against “fixed processes”. Fixed processes are a feature of ARM v4 or v5 architecture only; the following behaviour does not apply to devices based on the ARM v6 architecture.

Note that at time of writing few new devices are known to be using the moving model (or fixed processes). When ARMv5 support is dropped these caveats will no longer be relevant.

Non Execute-In-Place DLLs

For non-execute-in-place (non XIP) DLLs an additional code chunk is required for every fixed process which links against the DLL.

Imagine a 20Kb DLL (with a few bytes of WSD) that is loaded into four normal “non-fixed” processes, and two fixed processes. The (static) memory consumed is:

Code chunk shared by all moving processes = 20 Kb
Code chunk for each fixed process loading the DLL = 40 Kb
Data chunk for each process loading the DLL = 6×4Kb = 24 Kb

So to allow a few bytes of WSD there is a 64Kb increase in consumed memory! Note that the 20Kb code chunk shared by all processes is consumed whether or not WSD is enabled.

Execute-In-Place DLLs

For XIP DLLs there is no additional RAM cost other than the size of the WSD itself, rounded up to the next multiple of 4KB.

However:

  • An XIP DLL can be loaded by any non-fixed process, OR
  • it may be loaded by a single fixed process (and therefore cannot be loaded by any other processes whatsoever)

The ROM build fails if a DLL with static data links to a fixed process and any other process.

A Few Specific DLLs Cannot have WSD

DLLs that are required to initialise the file server cannot have WSD, e.g. HAL.DLL, EUSER.DLL, EFSRV.DLL.

Limit on the Number of DLLs in a Process with WSD

The number of DLLs with WSD in a process is limited only by the address space available for the writeable static data.

Fixed processes on the moving model (ARM v5 architecture) have an address space 1MB for WSD. Moving processes on the moving model, and all processes on the multiple model, have an address space of MIN(RAM_SIZE/2, 128MB), where RAM_SIZE is the total size of RAM on the device.

GCCE 3.4.3 Doesn't Support WSD Properly

There is a defect in GCCE 3.4.3 that means WSD is not properly supported on this compiler. See Symbian C++ Knowledgebase Q&As#Does Global writeable static data in DLLs work when projects are compiled with GCC-E v3.4.3? for more information.

Frequently Asked Questions

How Does Symbian’s WSD Implementation Work

For Moving Processes

Symbian OS supports a moving memory model, in which the data for a process is moved into a fixed virtual address space (the “run section”) when the process is active, and then back into the process’s “home” section when another process is active:

  • The data of processes in the “home section” is uniquely addressed both in terms of physical RAM pages, and in the MMU virtual address space. The data is protected from other processes, except for the Kernel.
  • The data of processes in the run section occupies the same virtual address space as all other processes when they run (the MMU moves the physical RAM pages into the appropriate virtual address space).

When a DLL with WSD is loaded (or at ROM build time), its code is fixed to point to static data at specific addresses. In order to ensure that only a single copy of the DLL code is required, Symbian’s moving-process WSD implementation ensures that the virtual addresses of static data is the same across every running process.

The way this works is:

  1. A specific address space is reserved for a per-process static data chunk. This chunk is to be used to hold all the static data for all the DLLs loaded into the process.
  2. At ROM build time, the kernel reserves specific addresses within the static data chunk for all the WSD in all the ROM loaded DLLs. The addresses for ROM-based DLLs are reserved from the top of the static data chunk address space to the bottom. Note that static data addresses for RAM based DLLs are reserved when the DLL is first loaded into any process. In this case, addresses are reserved from the bottom of the static data chunk address space.
  3. When the first DLL with WSD is loaded into a process, a static data chunk is created to hold the static data for all DLLs that are loaded into the process.
  4. Any global static data in the DLL is written to its specific reserved addresses. Note that addresses are reserved for that particular static data across all processes; if the DLL is loaded into another process, any static data will get the same virtual address.

For Fixed Processes

A fixed process is one in which the process data does not move; code is run on process data stored in the process “home section”.

Since the static data for every fixed process is uniquely addressed, and a DLL can only point to a single address for its data, the implication is that a separate copy of the DLL code is required for every fixed process that loads the DLL.

The restrictions listed in #ARM v4 & v5 architecture specific costs and limitations directly result.

  • For XIP based devices the DLL code chunk address is fixed at ROM build time, and there can only be one copy of the DLL code. Therefore the DLL code can address the data in either a single fixed process or the virtual address used by all moveable addresses.
  • For non-XIP based devices the DLL is run from RAM, and the loader is able to fix-up the address that a DLL expects its data at load time (rather than ROM build time). Therefore in this case the loader creates a separate copy of DLL code for each fixed process that loads the DLL and a single copy shared by all moving processes.

Notes:

Fixed processes are not supported or required on ARM v6 architectures. This discussion only applies to devices based on ARM v4 or v5. On ARM v6 architectures, each DLL with WSD has a reserved address, similar to the ARM v5 moving process case. However there is no ‘home section’ and memory is not relocated between low and high addresses on a context switch. Instead, each process uses its own set of page tables for the bottom half of virtual address space.

Why Does the Emulator Only Allow A DLL With WSD To Load Into A Single Process?

Note.pngNote: This section applies to Symbian^2 and earlier. From Symbian^3 WSD is transparently supported in the Emulator and WSD can be used in any number of processes.

The Symbian platform permits a DLL with global data to be loaded into only one process. This limitation is a result of the way the emulator is implemented on top of the Windows process model.

Separate Symbian OS processes are emulated within a single Windows process. To preserve Symbian OS semantics there should be one copy of the global data for each emulated process. However this is not possible since a DLL on the emulator is just a Windows DLL; Windows gives it a single copy of any global data it needs when it is loaded.

What Happens If EPOCALLOWDLLDATA Isn’t Declared For A DLL With WSD?

On The Emulator

Most constant data should be treated by the compiler as read-only rather than writeable static data (the exception is when the const data has a non-trivial constructor, so a real variable is required during initialisation).

Unfortunately, different compilers sometimes treat const data as WSD. For example, CodeWarrior puts it in writeable data and initialises it at run time. MSVC generally puts it into read-only data, but occasionally puts it into writeable data.

As most DLLs have const data, this means that the compilers have “accidentally” created WSD in almost every DLL. Symbian cannot therefore rigorously enforce the “single process can load a DLL with WSD” rule, as the emulator would not work.

On the emulator, the workaround is to recognise two types of DLL global data:

  • Deliberate global data is where the programmer specifies that they want DLL global data (using the epocallowdlldata keyword in the MMP file. In this case any global data in the DLL is assumed to be deliberate, and the “DLL loaded into one-emulated-process” rule applies.
  • Accidental global data is the data introduced by the compiler with no encouragement from the programmer. If epocallowdlldata is absent, global data is assumed to be accidental and the rule does not apply. Note that the global data includes both const and non-const static data; there is no way to tell the compilers to only apply it to non-const data – if we could do that then we could force correct handling of const static data.

In order to prevent abuse of this workaround there are restrictions on what can be done with accidental global data; specifically, the emulator will fault if any of the global data’s initialisers attempt to call kernel functions (i.e. executive calls).

Note that there is only one copy of the global data. Therefore it is possible for two processes to write to the same "accidental" global data (causing undefined behaviour). The “DLL loaded into one-emulated-process” rule prevents this being a problem for deliberate global data.

On Real Hardware

The compiler will fail the build with an error indicating that the code has initialised or un-initialised data. See [2] for information on how to locate the cause of these errors.

Can Kernel DLLs Have WSD

Yes, WSD is supported for kernel DLLs (through alternative mechanisms to those described in this paper)

Of course, kernel DLLs are guaranteed to be loaded into only one process, so the per-process multiplication of RAM usage does not apply. The platform will work correctly with global data in any kernel DLL.

See [3] for more information.

Emulator Support For WSD

Symbian^1 provides emulator WSD (EWSD), a mechanism to allow DLLs with WSD to be able to load into multiple processes.

The support is not transparent to the DLL writer; they have to wrap the WSD in a data structure and pass it to the EWSD library. This library maintains the relationship between the library, and its WSD values in each process.

The DLL writer has to make emulator-conditional changes to the source code to:

  • wrap all the WSD in the DLL into a single “wrapper” data structure
  • change code to refer to the wrapper data member rather than directly
  • prevent EPOCALLOWDLLDATA being declared in the emulator build

For example, consider a DLL with a function foo() that uses WSD iState as shown:

// source for foo.cpp
int iState;
void foo()
{
if (iState == ESomeState)
{
//do something
}
else
{
//do something else
}
}

You would change as shown:

// source for foo.cpp
struct MyWSD
{
int iState;
};
 
void foo()
{
#ifdef _USE_EWSD_
MyWSD *MyWSDPtr = Pls<MyWSD>(ThisLibUid);
// Pls is an API provided by the ewsd - it fetches the ptr for this
// (process,libuid) tuple.
// You can also pass an initialisation function to Pls() to initialise the WSD
// – initialisation is only done the 1st time the templated function is called
#endif
 
if (MyWSDPtr->iState == ESomeState)
{
//do something
}
else
{
//do something else
}
}

The MMP file of that DLL must not have EPOCALLOWDLLDATA for the emulator build, so you would make its specification conditional on use of EWSD in the MMP file:

#ifndef _USE_EWSD_
EPOCALLOWDLLDATA
#endif

If the DLL has a lot of WSD then recoding as above may take significant effort. Note that this recoding is not necessary if your DLL is only to be loaded into a single process, or if you only need to test on target.

Alternatives To Using Global Writeable Static Data

There are costs that may make using WSD unviable in a specific case – for example, using WSD may increase the RAM consumption significantly.

The following sections describe the alternatives that can be used to port code that makes use of global writeable static data.

Use Thread-local Storage (TLS)

Thread Local Storage (TLS) is a single per-thread word that can be used to simulate global writeable static data.

All the static data in the DLL is grouped into a single struct or class. On creation of the thread, an instance of the thread is allocated on the heap and a pointer to this data is saved to TLS (using Dll::SetTls()). On destruction of the thread the data is destroyed. Throughout the DLL the code references the TLS data (using Dll::Tls()) rather than the original global writeable static data. How this is done is described in detail in [4].

It is also possible to implement process-wide writeable global data using TLS. For more information see [5].

Wrap In A Server

Symbian OS supports writeable global static data in EXEs.

A common porting strategy is therefore to wrap the code in a Symbian server (which is an EXE), and expose its API as a client interface.

Note that it is also possible to use a server to make global data available to all threads at the cost of context switch to the server.

Move Global Variables Into Your Classes

With relatively small amounts of code, it may be possible to move most global data inside classes.

Build the Library as a Static Library

Where the code is only intended to be used directly by EXEs, you may be able to build it as a static LIB rather than as a DLL.

This approach can’t be used in a framework plug-in, or any other case where a DLL is a requirement of the architecture. Nor can it be used where the library is itself to be loaded by a DLL.

If the DLL code is likely to be shared by multiple processes it’s important to consider the potential code size and RAM usage costs of this approach.

Summary

WSD in DLLs is supported on the Symbian platform. It was provided primarily for ISVs porting code with WSD from other operating systems, where the cost of using the other alternatives is prohibitive.

Further Information

References

  1. Symbian C++ Knowledgebase Q&As#Don't be misled by const static data that isn't really const
  2. Symbian C++ Knowledgebase Q&As#How can I find the "uninitialised data" in my DLL?
  3. Symbian C++ Knowledgebase Q&As#Can I have writeable static data in my LDD, PDD, or kernel extension?
  4. Symbian C++ Knowledgebase Q&As#How can I use Thread Local Storage (TLS) to Maintain Static Data in a DLL?
  5. Symbian C++ Knowledgebase Q&As#Process local storage: How can I use TLS to implement writable global data which is shared between threads in the same process?

Glossary

Term Definition
Execute in place (XIP) On an XIP “ROM” the ROM memory is mapped into the process address space. Code is executed directly from the ROM. NOR flash is XIP memory.
Non XIP ROM is not mapped into the process address space. Code is loaded into RAM before being executed.
Chunk A collection of physical RAM pages mapped to contiguous virtual addresses. In EKA2 a process may have only 8 chunks.

Licence icon cc-by-sa 3.0-88x31.png© 2010 Symbian Foundation Limited. This document is licensed under the Creative Commons Attribution-Share Alike 2.0 license. See http://creativecommons.org/licenses/by-sa/2.0/legalcode for the full terms of the license.
Note that this content was originally hosted on the Symbian Foundation developer wiki.

This page was last modified on 25 August 2011, at 00:54.
35 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×