×
Namespaces

Variants
Actions

Fundamentals of Symbian C++/Streams And Stores

From Nokia Developer Wiki
Jump to: navigation, search
Article Metadata
Article
Created: hamishwillee (10 Jan 2011)
Last edited: hamishwillee (23 Jul 2012)

Contents

Streams

The Symbian C++ stream classes are used to serialize and deserialize objects, in other words, to convert their internal data into a series of bytes and to initialize them from a series of bytes.

The most obvious purpose of this is to provide a layer over the file system, enabling objects to be easily persisted into some nonvolatile storage medium.

Classes that support persistence will typically declare and define member functions with the following form:

void ExternalizeL(RWriteStream& aStream) const;
 
void InternalizeL(RReadStream& aStream);

ExternalizeL() will write the object’s internal state to the stream. InternalizeL() will initialize the object’s internal data from the stream.

Classes will often ensure that objects can only be initialized on construction by making InternalizeL() private and instead provide an overload of NewL() to construct new objects from the stream:

static CMyClass* NewL(RReadStream& aStream);

Although the most common function of streams is persistence, it is not the only one: streams are also used to serialize data into memory in preparation for transferring it across processes, and some streams transform the data, for example, by encrypting it.

To use streams and stores, link against ESTOR.lib. The header files declaring the classes described here are prefixed with 's32'.

Types of Stream

RReadStream and RWriteStream are abstract classes, and several subclasses are defined. What happens to the data after the object has written it to the stream depends on the concrete subtype of the stream parameter. It may be written:

  • directly to a file: RFileReadStream, RFileWriteStream
  • to a store: RStoreReadStream, RStoreWriteStream
  • into memory:
    • raw memory: RMemReadStream, RMemWriteStream
    • a fixed-size buffer: RDesReadStream, RDesWriteStream
    • a dynamic buffer: RBufReadStream, RBufWriteStream.

Some streams are initialized with other stream objects and effect a transformation on the data before sending it on:

  • Huffman coding: RHufDecodeReadStream, RHufEncodeWriteStream
  • encryption: RDecryptStream, REncryptStream

Initialization and Cleanup

Streams are typically initialized with an Open() method that takes as a parameter the destination for the data; the file stream is initialized with a file, the store stream with a store, and so on.

 void RDesReadStream::Open(const TDesC8& aDes)
 
TInt RFileReadStream::Open(RFs& aFs,const TDesC& aName,TUint aFileMode)
 
void RStoreReadStream::OpenL(const CStreamStore& aStore,TStreamId anId)

Streams that transform the data take the destination stream as a parameter, as well as any other parameters required for the transformation:

void RDecryptStream::OpenL(RReadStream& aHost,const CPBEncryptionBase& aKey)

For example, an REncryptStream could be initialized with an RFileStream, which is itself initialized with an RFile, and when my class externalizes its data, the data is encrypted before being stored in the file system:

Stream chaining

Write streams buffer the data that is written to them; any buffered data can be explicitly written out by calling CommitL(). Until CommitL() is called, the user is able to roll back any data written to the stream: this makes it possible to ensure that a series of objects can be written atomically.

Initialized streams own resources that must be cleaned up when the user has finished with the stream or in the event of a leave.

RReadStream and RWriteStream provide two cleanup methods:

  • Release() frees the stream’s resources.
  • Close() is identical to Release() if the stream is a read stream. If the stream is a write stream, it writes any buffered data to the stream, then frees the stream’s resources.

They also provide a PushL() member function, which calls CleanupReleasePushL().

The following code sample demonstrates initialization and cleanup:

RFs fs;
 
User::LeaveIfError(fs.Connect());
 
RFileWriteStream writer;
 
writer.PushL(); // call CleanupReleasePushL()
 
User::LeaveIfError(writer.Open(fs, KMyFile, EFileWrite));
 
// a series of writes
 
// if any of them fail, Release() will be called,
 
// and nothing will be written out
 
writer.CommitL(); // all buffered data is written out
 
CleanupStack::PopAndDestroy(&writer); // release resources

Internalizing and Externalizing

The stream interface defines methods for reading and writing the following data types:

  • 8- and 16-bit character data
  • 8-, 16-, 32-bit signed and unsigned integers
  • 32- and 64-bit real numbers

Classes use these to externalize and internalize basic data types. Note that the size of the types must be specified explicitly. It is not possible to write a TInt as a TInt because its size is platform-dependent. It must be written as a TInt8, TInt16 or TInt32. To save space, classes should choose the smallest type that is guaranteed to fit.

When more than one element is externalized, the data must be internalized in the same order in which it was externalized.

class TMyClass
{
public:
void ExternalizeL(RWriteStream& aStream) const;
void InternalizeL(RReadStream& aStream);
 
private:
TMyEnumerationType iEnumType;
TInt32 iIntegerType;
TBool iBooleanType;
};
 
void TMyClass::ExternalizeL(RWriteStream& aStream) const
{
aStream.WriteInt8L(iEnumType);
aStream.WriteInt32L(iIntegerType);
aStream.WriteInt8L(iBooleanType);
}
 
void TMyClass::InternalizeL(RReadStream& aStream)
{
iEnumType = TMyEnumerationType(aStream.ReadInt8L());
iIntegerType = aStream.ReadInt32L();
 
iBooleanType = aStream.ReadInt8L();
}

To help preserve data compatibility it is also a good idea to include a version number in an externalized data structure so that new versions of the application can read older streams.

The Stream Operators

The stream framework defines the operators << and >> for writing to and reading from a stream:

writeStream << object;
readStream >> object;

The stream framework provides the implementation to make these operators externalize and internalize many of the built-in types and many basic Symbian C++ classes.

If invoked on an instance of any class that defines ExternalizeL() and InternalizeL() methods, the operators will call them.

class TMyContainerClass
{
public:
void ExternalizeL(RWriteStream& aStream) const;
void InternalizeL(RReadStream& aStream);
private:
TMyClass iUserDefinedDataMember;
};
 
void TMyContainerClass:: ExternalizeL(RWriteStream& aStream) const
{
aStream << iUserDefinedDataMember;
}
 
void TMyContainerClass:: InternalizeL(RReadStream& aStream)
{
aStream >> iUserDefinedDataMember;
}

Because these operators call leaving functions they can themselves leave, and are a very rare example of a leaving function without a trailing 'L'.

Externalizing and Internalizing Descriptors

The stream framework implements the stream operators for descriptors, and these should be used to externalize and internalize descriptors in preference to the ReadL() and WriteL() overloads in the stream classes themselves. The operators write out the length of the descriptor automatically, and apply compression to unicode strings.

class TMyClass
{
public:
void ExternalizeL(RWriteStream& aStream) const;
void InternalizeL(RReadStream& aStream);
private:
TBuf<32> iMyBuffer;
};
 
void TMyClass::ExternalizeL(RWriteStream& aStream) const
{
aStream << iMyBuffer;
}
 
void TMyClass::InternalizeL(RReadStream& aStream)
{
aStream >> iMyBuffer;
}

To initialize dynamic descriptors from a read stream, use the overloads of HBufC::NewL() or RBuf::CreateL():

class CMyClass : public CBase
{
public:
void ExternalizeL(RWriteStream& aStream) const;
void InternalizeL( RReadStream& aStream);
 
private:
RBuf iMyRBuf;
HBufC* iMyHBufC;
};
 
void CMyClass::ExternalizeL(RWriteStream& aStream) const
{
aStream << iMyRBuf;
aStream << *iMyHBufC;
}
 
void CMyClass::InternalizeL(RReadStream& aStream)
{
iMyRBuf.CreateL(aStream, KMaxTInt);
iMyHBufC = HBufC::NewL(aStream, KMaxTInt);
}

The aMaxLength argument to these methods is confusing. The aMaxLength arguments supplied to some of the other overloads of HBufC16::NewL() and RBuf16::CreateL() specify the size of the buffer to create, so it looks as if the code fragment above will create a descriptor with a size of KMaxTInt.

However, the externalized descriptor knows its size, and the descriptor constructed from it will never be bigger than that. In this case, the aMaxLength argument specifies the maximum amount of data to read from the stream; if we know the target should never be bigger than 32 16-bit values, we can guarantee that by supplying 32 here:

Length of externalized buffer Value for aMaxLength Length of internalized buffer
128 32 32
128 KMaxTInt 128

Stores

A store is a collection of streams in which each stream is accessed by an identifier of type TStreamId. The abstract base class for all stores is CStreamStore, and the store component defines a hierarchy of subclasses:

Cstreamstore class hierarchy.png

CBufStore is a collection of streams stored in memory. CSecureStore encrypts all its streams, and CEmbeddedStore is contained within, and can be managed as, a single stream. These are all fairly specialized uses of the store component; most stores are one of the two types of file store.

CStreamStore

The base class CStreamStore defines:

  • an interface to add streams to, and delete streams from, a store
  • a mechanism to commit changes to a store, or revert uncommitted changes
  • space compaction and reclamation.

CPersistentStore

The persistent store CPersistentStore class adds functions to set and retrieve the root stream ID. This is the place where users of the store start, and it typically contains a list of the stream IDs of other streams in the store, enabling the user to find the relevant stream efficiently. The diagram below shows a root stream containing four stream IDs that point to the four streams containing the data:

Root stream.png

CFileStore

A file store is a store whose underlying persistent storage is a file. It is possible to open an existing file store using CFileStore, but to create a new file store you must use the creation functions of CPermanentFileStore and CDirectFileStore.

CDirectFileStore and CPermanentFileStore

The difference between direct and permanent file stores is that, once a stream has been committed to a direct file store, the stream cannot be deleted, replaced, extended or changed. If a single stream needs to be changed, the entire store must be rewritten. By contrast, streams inside permanent file stores can be changed and replaced.

Direct file stores are useful for applications where the data is typically all saved together. For example, a text-editing application will typically treat a saved document as a single unit. When the user edits the document and saves it, the application will replace the entire stored copy rather than individual pieces.

Permanent file stores are useful for applications that need to update individual components of their stored data without rewriting the whole thing. For example, a contacts application will want to be able to add, remove and edit individual contacts without replacing the entire database.

Initialization and Cleanup

The functions for creating a new file store or opening an existing one are the same for permanent and direct file stores. The first four functions are almost exactly the same as those for creating a new file:

static CDirectFileStore* CDirectFileStore::OpenL(
 
RFs& aFs,const TDesC& aName,TUint aFileMode)
 
static CDirectFileStore* CDirectFileStore::CreateL(
 
RFs& aFs,const TDesC& aName,TUint aFileMode)
 
static CDirectFileStore* CDirectFileStore::ReplaceL(
 
RFs& aFs,const TDesC& aName,TUint aFileMode)
 
static CDirectFileStore* CDirectFileStore::TempL(
 
RFs& aFs,const TDesC& aName,TUint aFileMode)

The last two enable you to create a new file store from an open file handle:

static CDirectFileStore* CDirectFileStore::FromL(RFile& aFile)
 
static CDirectFileStore* CDirectFileStore::NewL(RFile& aFile)

Before actually using the store, it is essential to set the store’s ID. At the very least, you must set the first ID value to one of KDirectFileStoreLayoutUid or KPermanentFileStoreLayoutUid:

store->SetTypeL(TUidType(KDirectFileStoreLayoutUid,
 
KUidAppDllDoc, KMyAppUid));

The second and third UID values may be set to KNullUid, but setting them can help the application check that it is opening the correct file.

Before destroying the store, call CommitL() to make sure any data is written out to the underlying file, then clean up the store by deleting it or calling CleanupStack::PopAndDestroy() if it was on the cleanup stack.

Creating a Store Write Stream

Given an open file store, you can create a write stream:

RStoreWriteStream stream;
 
TStreamId id = stream.CreateLC(*store);

This function returns the stream ID. If this stream is not to be the root stream, its stream ID should be written to another stream so it can be retrieved later. If it is to be the root stream, it can be set using SetRootL():

store->SetRootL(id);

For example, the following code stores two instances of CMyClass each in their own stream, then writes the stream IDs into a third stream, which is set as the root:

// create two instances of CMyClass
 
CMyClass* myClass1 = CMyClass::NewLC(1);
 
CMyClass* myClass2 = CMyClass::NewLC(2);
 
// initialize the file server session
 
RFs fsSession;
 
User::LeaveIfError(fsSession.Connect());
 
CleanupClosePushL(fsSession);
 
// create a new file store
 
TUint shareMode = EFileShareExclusive|EFileWrite;
 
CDirectFileStore* store = CDirectFileStore::CreateLC(fsSession, KFileName, shareMode);
 
store->SetTypeL(KDirectFileStoreLayoutUid);
 
RStoreWriteStream writer;
 
// externalize myClass1 to a stream
 
TStreamId writeStream1 = writer.CreateLC(*store);
 
writer << *myClass1;
 
writer.CommitL();
 
CleanupStack::PopAndDestroy(&writer);
 
// externalize myClass2 to another stream
 
TStreamId writeStream2 = writer.CreateLC(*store);
 
writer << *myClass2;
 
writer.CommitL();
 
CleanupStack::PopAndDestroy(&writer);
 
// write the stream IDs to a third stream,
 
// and set that as the root
 
TStreamId root = writer.CreateLC(*store);
 
writer << writeStream1;
 
writer << writeStream2;
 
writer.CommitL();
 
CleanupStack::PopAndDestroy(&writer);
 
store->SetRootL(root);
 
// commit and clean up
 
store->CommitL();
 
CleanupStack::PopAndDestroy(4, myClass1);

Creating a Store Read Stream

Given an open file store and the ID of a stream in it, you can create a read stream. For example, to open the store’s root stream:

RStoreReadStream rootStream;
 
rootStream.OpenLC(*store, store->Root());

Given that streams inside stores are accessed by their ID, it should be apparent that if you lose a stream’s ID then the stream becomes inaccessible and the memory for it is leaked.

The following code opens the store created in the previous example, reads the stream IDs for the two objects from the root stream, then initializes the two objects from their streams:

// initialize file server session
 
RFs fsSession;
 
User::LeaveIfError(fsSession.Connect());
 
CleanupClosePushL(fsSession);
 
// open the file store
 
TUint shareMode = EFileShareExclusive|EFileWrite;
 
CDirectFileStore* store = CDirectFileStore::OpenLC(fsSession, KFileName, shareMode);
 
// open the root stream
 
TStreamId rootId = store->Root();
 
RStoreReadStream reader;
 
reader.OpenLC(*store, rootId);
 
// read the stream IDs for our objects
 
TStreamId writeStream1;
 
reader >> writeStream1;
 
TStreamId writeStream2;
 
reader >> writeStream2;
 
CleanupStack::PopAndDestroy(&reader);
 
// internalize the first object from the first stream
 
reader.OpenLC(*store, writeStream1);
 
CMyClass* myClass1 = CMyClass::NewL(reader);
 
CleanupStack::PopAndDestroy(&reader);
 
CleanupStack::PushL(myClass1);
 
// internalize the second object from the second stream
 
reader.OpenLC(*store, writeStream2);
 
CMyClass* myClass2 = CMyClass::NewL(reader);
 
CleanupStack::PopAndDestroy(&reader);
 
CleanupStack::PushL(myClass2);
 
// clean up
 
CleanupStack::PopAndDestroy(4, &fsSession);

Swizzles

The templated swizzle classes TSwizzleC<class T> and TSwizzle<class T> provide a dual representation of an object as a stream ID, if the object is not yet loaded into memory, or a pointer, if it is.

This enables objects to implement deferred loading. They can keep data members on disk unless they are actually needed, and load them into memory on demand.

For example, a word-processing application could use swizzles to defer loading of embedded pictures, so the document is fast to load and memory-efficient, and pictures are loaded only as the user scrolls through the document, bringing them into view.

See also: Symbian Store Streams Guide


Licence icon cc-by-sa 3.0-88x31.png© 2010 Symbian Foundation Limited. This document is licensed under the Creative Commons Attribution-Share Alike 2.0 license. See http://creativecommons.org/licenses/by-sa/2.0/legalcode for the full terms of the license.
Note that this content was originally hosted on the Symbian Foundation developer wiki.

This page was last modified on 23 July 2012, at 10:47.
51 page views in the last 30 days.

Was this page helpful?

Your feedback about this content is important. Let us know what you think.

 

Thank you!

We appreciate your feedback.

×