×

Discussion Board

Results 1 to 7 of 7
  1. #1
    Regular Contributor
    Join Date
    Dec 2005
    Posts
    59

    Audio PCM Frame Size Implications

    Please excuse this repost from a year ago, but I thought that maybe with the passage of time, someone has a better analysis of this situation, which has prevented me from porting my application to Symbian phones:...

    The subject of frame size in PCM streaming audio has come up from time to time in this forum, as well as in various example programs. After doing some empirical studies on this subject using a Nokia 6620, I have concluded something quite different from the conventional wisdom regarding this question.

    I had assumed that Series 60, 2nd Edition devices used a native frame size of 320 bytes (160 samples). I began developing an application that would analyze sound in real-time. Using a frame size of 160 samples, I began to notice that on every 13th frame, the last 32 samples were unfilled when the buffer was returned. Initially I found this out by pre-filling each buffer with the number 11111 in every element just before calling ReadL. Since the acoustic environment was quiet during my testing, this value would never occur naturally. But as the buffers were received by MaiscBufferCopied, every 13th buffer was found to contain 11111 in the last 32 slots. I subsequently confirmed this more directly by examining the Length() of the returned descriptor. I found it suspicious that 13 x 160 - 32 = 2048, a nice power of 2.

    I began to experiment with frame sizes of 128, 132, 150, 180, and 256 samples. In the cases of 128 and 256, the buffers were always filled completely. But for all the other frame sizes, the buffers were filled in a manner that suggested that things were forced to come out even at 2048 samples by leaving some portion of occasional buffers only partially filled. For example, with a frame size of 180, every 12th buffer was unfilled in the last 112 slots. 12 x 180 - 112 = 2048. As long as you use the Length() of the returned descriptor to process the returned buffer, and not just assume it is full, gap-free streaming is still possible. However, it is simpler and more efficient to use a frame size that does not require any shortened buffers. On the Nokia 6620, that means divisors of 2048.

    The special role of 2048 samples is more than just a numerological oddity. It also turns out to set a limit on the latency in getting the audio data into your application. I tested the hypothesis that audio blocks of 2048 samples were developed in their entirety by the system audio drivers before any of that data is delivered to the application. I used User::TickCount() within my MaiscBufferCopied callback to timestamp the arrival of the buffers. It turns out that with a frame size of 256 samples, 8 frames are delivered as fast as MaiscBufferCopied can take them. Then, based on the timestamps, there is a delay of about 256 msec. before the next group of 8 frames is delivered (I am using a sample rate of 8000 samples per second). When the first of those 8 frames is returned, it is already almost 256 msec. old. In my application, I need to display a real-time analysis of the sound in less time than that, so this latency is intolerable. As long as the system insists on accumulating 2048 samples before giving any of them to me, the latency problem will always remain. About the only thing I can do is to try a faster sample rate so that 2048 samples will not take quite so long. But that option is not available on the Nokia 6620 and many other phones because the only input sample rate for PCM that is valid is 8000 samples per second.

    If anyone is also working on real-time audio analysis with streaming input and you would like to see the code that I used in these studies, just send me a message through the forum.

    Robert Scott
    Ypsilanti, Michigan

  2. #2
    Registered User
    Join Date
    Dec 2006
    Posts
    2,280

    Re: Audio PCM Frame Size Implications

    Hi,

    Yes I think you are correct in your analysis. I haven't worked on the audio driver in any S60 phones so I can't give you a definitive answer (and probably wouldn't be allowed even if I had).

    4096 bytes or 2048 samples for PCM16 seems to be the native buffer size used by the audio device. I suspect this is hardware determined but even if that isn't the case there is a trade off between buffer size and performance (CPU usage and hence power consumption). The larger the buffer the less work you do. I expect the processor gets an interrupt to say that another buffer is ready to go and then it copies it.

    So, I think the only way you can reduce the latency is to increase the sample rate. I think newer devices support higher sample rates so you could perhaps support from S60 3rd Edition onwards only - that wouldn't be such a terrible policy going forwards anyway since the older devices have relatively small sales volumes compared to the newer ones.

    Sorcery

  3. #3
    Regular Contributor
    Join Date
    Dec 2005
    Posts
    59

    Re: Audio PCM Frame Size Implications

    Thanks for the info. I will consider making S60 3rd edition as the minimum system requirements.

    My application is currently running on Pocket PCs with a 22,050 sample rate. The Pocket PCs I have tested have different system buffer sizes, but they all are less than 750 samples (34 msec.). Even if some device uses 2048 samples, the latency will be 92 msec, which is a little marginal. 4096 would be intolerable.

    So if I do port my application to Symbian phones, customers are going to be asking me which phones will work so they will know whether to buy it. I cannot afford to buy one of every model phone that comes out, so I need to rely on published specs. If the system buffer size is a secret that manufacturers refuse to devulge, then I will be very limited in my ability to recommend specific phones.

    Robert Scott
    Real-Time Specialties

  4. #4
    Registered User
    Join Date
    Dec 2006
    Posts
    2,280

    Re: Audio PCM Frame Size Implications

    You should be able to query the buffer size from CMMFDevSound (can't remember which function, something like Config or Capabilities). You could write a little app that prints it to the screen and then use the Remote Device Access service to test on multiple devices.

    Sorcery

  5. #5
    Regular Contributor
    Join Date
    Dec 2005
    Posts
    59

    Re: Audio PCM Frame Size Implications

    I do not have the devices to test on. I cannot afford to buy them all. The manufactures will not lend them to me. My customers do not have them yet. That is why they will be asking me before they buy their phone, "Will the Nokia xxxx model work on your application?"

    Robert Scott
    Ypsilanti, Michigan

  6. #6
    Registered User
    Join Date
    Dec 2006
    Posts
    2,280

    Re: Audio PCM Frame Size Implications

    That's why I suggested the Remote Device Access (RDA) service. You probably couldn't test your application on it completely since I don't think there's a way to inject audio to the microphone (although it would be quite easy to offer the service by allowing you to upload a wav file to the host PC I suspect).

    Anyway, RDA allows you to install your applications to a wide selection of devices to test them remotely. You could use it to query the buffer sizes and determine supported models. You could also extrapolate from tested devices because the Nokia devices come in families which share the same hardware (and hence the same base port -> same buffer sizes for audio).

    Sorcery

  7. #7
    Regular Contributor
    Join Date
    Dec 2005
    Posts
    59

    Re: Audio PCM Frame Size Implications

    Quote Originally Posted by Sorcery-ltd View Post
    That's why I suggested the Remote Device Access (RDA) service...
    Sorcery
    Wow, that is neat. I didn't realize what RDA meant, but after looking at it a while, it does appear to offer just what I need. Thanks.

    Robert Scott
    Ypsilanti, Michigan

Similar Threads

  1. Streaming audio input frame size implications
    By tunelabguy in forum Symbian Media (Closed)
    Replies: 6
    Last Post: 2008-10-21, 14:40
  2. Converting AMR buffer to PCM - low audio quality output :-(
    By coolbreez in forum Symbian Media (Closed)
    Replies: 10
    Last Post: 2006-10-09, 05:13
  3. Audio Input Stream on N80
    By tkaihock in forum Symbian Media (Closed)
    Replies: 9
    Last Post: 2006-06-30, 09:15
  4. Please HELP, very strange...
    By gg17 in forum Mobile Java General
    Replies: 1
    Last Post: 2005-07-21, 10:43
  5. 编译通过运行出错,请高手指教!
    By hdmxch in forum [Archived] Other Programming Discussion 关于其他编程技术的讨论
    Replies: 9
    Last Post: 2004-08-08, 14:36

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×