×

Discussion Board

Results 1 to 14 of 14
  1. #1
    Registered User
    Join Date
    Sep 2008
    Posts
    8

    The issue of "OutputStream.flush()"/Memory leak.

    Hi All.

    I try to upload a file with "POST" method by HttpConnection, I find the memory is on the decrease down quickly while executing OutputStream.flush(). Especially, for the file with large size,for example the size is more than 2MB. Some consumed memory will not be released after uploading, and the value of System.getProperty("com.nokia.memoryramfree") will be decreased down after uploading for every time. In this situation, after uploading the file for several times, the MidLet App will be crashed by outOfMemoryError

    This issue is found on Nokia E51,and the emulator(S60_3rd_FP1/S60_3rd_FP2).

    The corresponding codes below:

    PHP Code:
    try {
        
    cn = (javax.microedition.io.HttpConnectionConnector.open(tmpUrl);
        
    cn.setRequestMethod("POST");
        
    Enumeration em requestProperty.keys();
        while ( 
    em.hasMoreElements()) {
            
    String key = (String) em.nextElement();
            
    String value = (String) requestProperty.get(key);
            
    cn.setRequestProperty(keyvalue);
        }

        
    OutputStream os cn.openOutputStream();
        
    /*
        bs: byte[].
        Here,the corresponding parameters are all in the it.
        */
        
    os.write(bs);
        
    bs null;
     
        
    InputStream is file.getInputStream();
        
    int len = -1;
        
    bs = new byte[4096];
        while ( (
    len is.read(bs)) != -1) {
            
    os.write(bs0len);
        }
        
    bs null;
        
    is.close();
        
    is null;

        
    // closing boundary marker
        
    write(osnewline);
        
    write(osdashdash);
        
    write(osmultipartBoundaryBits);
        
    write(osdashdash);
        
    write(osnewline);

        
    os.flush();
        
    os.close();

        
    resultCode cn.getResponseCode();
        if ( 
    resultCode == HttpConnection.HTTP_MOVED_TEMP || resultCode == HttpConnection.HTTP_MOVED_PERM
             
    || resultCode == HttpConnection.HTTP_SEE_OTHER
             
    || resultCode == HttpConnection.HTTP_TEMP_REDIRECT) {
     
             ...
        }
    }
    catch (
    Exception e) {
        break;
    }catch (
    OutOfMemoryError o){
        break;
    }
    finally{
        
    cn.close();
        
    cn null;
        
    System.gc();

    For example, I try uploading a file with 2.3MB size.
    Before executing the code os.flush() , I print below values:
    HTML Code:
    Runtime.getRuntime().totalMemory()            = 12273920
    System.getProperty("com.nokia.memoryramfree") = 7643136   
    After executing the code os.flush(), I print the values again:
    HTML Code:
    Runtime.getRuntime().totalMemory()            = 10607616
    System.getProperty("com.nokia.memoryramfree") = 6709248
    So, there is a difference:
    HTML Code:
       (12273920 - 10607616) + (7643136- 6709248) 
              1666304        +       933888         = 2600192 (about 2.5M)
    This 2.5MB size is instability, sometimes it is more(or less) than 2.5M.
    And the 2.5MB memory will NEVER be collected by the phone after executing the code os.flush(),whether uploading successfully or not , unless I quit the Midlet App.

    Theoretically, the 2.5MB size memory will be collected after executing below codes,but actually, it doesn't.
    HTML Code:
    os.flush();
    os.close();
    os = null;
    Is there anybody know how/why, or other solutions?
    Whether the method OutputStream.flush() will cause the MEMORY LEAK when the size of OutputStream is big?

    Thanks in advance.



    With best regards.
    Last edited by hreny; 2009-03-19 at 08:41.

  2. #2
    Registered User
    Join Date
    Sep 2007
    Location
    Bangalore
    Posts
    868

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Hi hreny ,
    If your are uploading the big files in j2me you need to plan how to post that big data properly to the out put stream . In your code to post 2.3MB data you are preparing the big bite buffer of size 2.3MB and you are writing that data at a time to the output stream . Storing the entire data in to the bite buffer will cause the out of memory error. And in your code after writing your data you are not freeing up that 2.3MB bite buffer . in finally you can do bs = null ;

    And one more thing to avoid the out of memory when you are uploading the big files is

    // Open the output stream
    // Create a bite buffer of 1KB
    // Read your files in the multiples of 1KB and write in the out put stream at multiple time till your file data complets
    // At the end you can do flush or you can take the connection.getResponseCode(); which will atometiclly flush your data. and give you the status.

    Hope this will helps you

  3. #3
    Registered User
    Join Date
    Sep 2008
    Posts
    8

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Hi bhanuchandar.k ,

    First of all, Thanks for your reply.
    I am sorry, I had posted the codes before were simplified. Now, I have already updated it.
    My codes are closing to what you said.

    And I also have tried the buffer with 1KB(modifies 4096 to 1024) size base on your idea, but the result is the same. The value of System.getProperty("com.nokia.memoryramfree") is still decreased down after uploading.

    Actually, the method os.flush() is the place where the "OutOfMemoryError?" happens very easy.

    It throws the below error:
    HTML Code:
    java.lang.OutOfMemoryError
    	at java.io.ByteArrayOutputStream.toByteArray(Unknown Source)
    	at com.symbian.midp.io.protocol.http.HttpConnectionNative.sendRequest(HttpConnectionNative.java:972)
    	at com.symbian.midp.io.protocol.http.HttpConnectionNative.ensureResponse(HttpConnectionNative.java:790)
    	at com.symbian.midp.io.protocol.http.HttpConnectionNative.outputStreamFlushed(HttpConnectionNative.java:708)
    	at com.symbian.midp.io.protocol.http.HttpConnectionNative$PlainDataOutputStream.flush(HttpConnectionNative.java:1190)
    	at com.opusmicro.io.HttpConnection.connect(Unknown Source)
    	at com.sun.me.web.request.Request.doHTTP(Unknown Source)
    	at com.sun.me.web.request.Request.run(Unknown Source)
    	at java.lang.Thread.run(Unknown Source)
    I appreciate your reply, Please give me more help.

    Thank you very much.

  4. #4
    Registered User
    Join Date
    Sep 2007
    Location
    Bangalore
    Posts
    868

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Hi,
    Don't do the flush at the end do like this
    loop
    //Write 4kb data
    //flush
    end loop
    see what will happen

  5. #5
    Registered User
    Join Date
    Sep 2008
    Posts
    8

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Hi bhanuchandar.k,

    Quote Originally Posted by bhanuchandar.k View Post
    Hi,
    Don't do the flush at the end do like this
    loop
    //Write 4kb data
    //flush
    end loop
    see what will happen
    I had tried what you said before.
    Server side can't get the whole data in one session after the first flush(),So Server side will not find the file data, and judge it as wrong data.

    Thanks.

  6. #6
    Registered User
    Join Date
    Mar 2010
    Posts
    2

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Hello,

    I have the same problem - OutputStream causes memory leaks. For example: after sent
    50 KB I lost about 26 KB !!! Runtime.getRuntime().freeMemory() dosen't show any
    memory lost. All is ok according it but System.getProperty("com.nokia.memoryramfree")
    indicates decreasing memory after each http request.

    Is anybody solved this problem ?

    Thanks in advance,
    Axel

  7. #7
    Registered User
    Join Date
    Apr 2010
    Posts
    15

    Re: The issue of "OutputStream.flush()"/Memory leak.

    +1, I have the same problem. This leak also occurs on the Sun WTK emulator.

    Consider the following loop:


    Code:
    HttpConnection connection;
    		OutputStream outStream;
    		byte[] sendData = new byte[1024];
    		int i = 0;
    		while (true) {
    				try {
    					i++;
    					System.out.println("Request number: " + i);
    					connection = (HttpConnection)Connector.open("http://someaddress/"); //openoutputstream?
    					connection.setRequestMethod(HttpConnection.POST);
    					connection.setRequestProperty("User-Agent", "SkyNet Mobile Client");
    					connection.setRequestProperty("Content-Type", "application/octet-stream");
    					connection.setRequestProperty("Content-Length", String.valueOf(sendData.length));
    					outStream = connection.openOutputStream();
    					outStream.write(sendData);
    					outStream.flush();
    					outStream.close();
    					outStream = null;
    
    					//perhaps read the response?
    					connection.close();
    				}
    				catch (Exception e) {
    					e.printStackTrace();
    				}
    				finally {
    					Thread.yield();
    				}
    
    		}
    This will eventually fail. If you watch the memory monitor, usage will go up and down due to garbage collection, but seems fine. However, there is a leak in the background. After a bit over 1000 requests (on a bare bone app, on the app I'm developing I can only do a bit over 100 with 512Kb heap size) I get this:

    Code:
    Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
            at java.lang.Object.clone(Native Method)
            at sun.java2d.SunGraphics2D.clone(Unknown Source)
            at sun.java2d.SunGraphics2D.create(Unknown Source)
            at javax.swing.JComponent.paint(Unknown Source)
            at java.awt.GraphicsCallback$PaintCallback.run(Unknown Source)
            at sun.awt.SunGraphicsCallback.runOneComponent(Unknown Source)
            at sun.awt.SunGraphicsCallback.runComponents(Unknown Source)
            at java.awt.Container.paint(Unknown Source)
            at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
            at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
            at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
            at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
            at java.awt.event.InvocationEvent.dispatch(Unknown Source)
            at java.awt.EventQueue.dispatchEvent(Unknown Source)
            at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
            at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
            at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
            at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
            at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
            at java.awt.EventDispatchThread.run(Unknown Source)
    Exception in thread "Thread-8" java.lang.OutOfMemoryError: Java heap space
            at java.util.Vector.<init>(Unknown Source)
            at java.util.Vector.<init>(Unknown Source)
            at java.util.Vector.<init>(Unknown Source)
            at javax.swing.tree.DefaultMutableTreeNode.insert(Unknown Source)
            at javax.swing.tree.DefaultMutableTreeNode.add(Unknown Source)
            at com.sun.kvem.memorymon.AllocationTree.enter(Unknown Source)
            at com.sun.kvem.memorymon.MemoryMonitor.enterMethod(Unknown Source)
            at com.sun.kvem.memorymon.MemoryListener.parseNextCommand(Unknown Source)
            at com.sun.kvem.memorymon.MemoryListener.run(Unknown Source)
    Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
            at java.awt.image.DataBufferInt.<init>(Unknown Source)
            at java.awt.image.Raster.createPackedRaster(Unknown Source)
            at java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown Source)
            at sun.awt.image.SunVolatileImage.getBackupImage(Unknown Source)
            at sun.awt.image.VolatileSurfaceManager.getBackupSurface(Unknown Source)
            at sun.awt.image.VolatileSurfaceManager.initialize(Unknown Source)
            at sun.awt.image.SunVolatileImage.<init>(Unknown Source)
            at sun.awt.image.SunVolatileImage.<init>(Unknown Source)
            at java.awt.GraphicsConfiguration.createCompatibleVolatileImage(Unknown Source)
            at java.awt.GraphicsConfiguration.createCompatibleVolatileImage(Unknown Source)
            at javax.swing.RepaintManager.getVolatileOffscreenBuffer(Unknown Source)
            at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
            at javax.swing.BufferStrategyPaintManager.paint(Unknown Source)
            at javax.swing.RepaintManager.paint(Unknown Source)
            at javax.swing.JComponent.paint(Unknown Source)
            at java.awt.GraphicsCallback$PaintCallback.run(Unknown Source)
            at sun.awt.SunGraphicsCallback.runOneComponent(Unknown Source)
            at sun.awt.SunGraphicsCallback.runComponents(Unknown Source)
            at java.awt.Container.paint(Unknown Source)
            at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
            at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
            at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
            at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
            at java.awt.event.InvocationEvent.dispatch(Unknown Source)
            at java.awt.EventQueue.dispatchEvent(Unknown Source)
            at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
            at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
            at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
            at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
            at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
            at java.awt.EventDispatchThread.run(Unknown Source)
    Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
    Warning: Could not start new emulator process:
      java.lang.NullPointerException
    It says it occurred in AWT threads, but that's no clue as it happens whenever there's too little memory left and whatever sorry guy is allocating memory gets the blame. Memory monitor shows me that I've got plenty of free memory. If I'm lucky I might get a scream from the VM:

    Code:
    #
    # An unexpected error has been detected by Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d89f971, pid=5500, tid=3464
    #
    # Java VM: Java HotSpot(TM) Client VM (10.0-b23 mixed mode, sharing windows-x86)
    # Problematic frame:
    # V  [jvm.dll+0xdf971]
    #
    This is on Sun's emulator, however Nokia shows the exact same behaviour. I've only tested on S60 so far. I can't see any other explanation than a memory leak behind the scenes (the app even closes with POSIX error codes), but question is, how do we get around them?
    Last edited by heldal; 2010-04-22 at 08:57.

  8. #8
    Super Contributor
    Join Date
    Mar 2008
    Location
    The Capital of INDIA
    Posts
    4,328

    Re: The issue of "OutputStream.flush()"/Memory leak.

    [QUOTE=heldal;728149]+1, I have the same problem. This leak also occurs on the Sun WTK emulator.

    Consider the following loop:


    Code:
    HttpConnection connection;
    		OutputStream outStream;
    		byte[] sendData = new byte[1024];
    		int i = 0;
    		while (true) {
    				try {
    					i++;
    					System.out.println("Request number: " + i);
    					connection = (HttpConnection)Connector.open("http://skynet.tele2.no/services/addFootage/"); //openoutputstream?
    					connection.setRequestMethod(HttpConnection.POST);
    					connection.setRequestProperty("User-Agent", "SkyNet Mobile Client");
    					connection.setRequestProperty("Content-Type", "application/octet-stream");
    					connection.setRequestProperty("Content-Length", String.valueOf(sendData.length));
    					outStream = connection.openOutputStream();
    					outStream.write(sendData);
    					outStream.flush();
    					outStream.close();
    					outStream = null;
    
    					//perhaps read the response?
    					connection.close();
    				}
    				catch (Exception e) {
    					e.printStackTrace();
    				}
    				finally {
    					Thread.yield();
    				}
    
    		}
    Hey welcome here on Forum Nokia Discussion Boards!!!!
    Is is better to start a new thread instead continue with the older one.

    Yes this will cause you the unexpected and expected exceptions/errors the reason is that code inside the infinite loop.You can not imagine that how many HTTP connection object will be created since you are working in a wrong style.

    Just make sure that the connection opening code executes exactly when you have asked it to execute and just once.you can put the system.out.println's and can check that.

    Please remove that code for opening the connection from the loop and I am sure that the issue will be solved.
    Last edited by raj_J2ME; 2010-04-22 at 09:32.
    Thanks with Regards,

    R a j - The K e r n e l


    Join Delhi-NCR Nokia Developer's Community,

  9. #9
    Registered User
    Join Date
    Apr 2010
    Posts
    15

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Quote Originally Posted by raj_J2ME View Post
    Hey welcome here on Forum Nokia Discussion Boards!!!!
    Is is better to start a new thread instead continue with the older one.

    Yes this will cause you the unexpected and expected exceptions/errors the reason is that code inside the infinite loop.You can not imagine that how many HTTP connection object will be created since you are working in a wrong style.

    Just make sure that the connection opening code executes exactly when you have asked it to execute and just once.you can put the system.out.println's and can check that.

    Please remove that code for opening the connection from the loop and I am sure that the issue will be solved.
    If I remove the opening code from the loop, then how am I supposed to perform more than one request?

  10. #10
    Super Contributor
    Join Date
    Mar 2008
    Location
    The Capital of INDIA
    Posts
    4,328

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Quote Originally Posted by heldal View Post
    If I remove the opening code from the loop, then how am I supposed to perform more than one request?
    That depends upon the situation.
    What do you mean by the multiple request?
    Why dont you put the request on some event like keypressed() or command?
    Thanks with Regards,

    R a j - The K e r n e l


    Join Delhi-NCR Nokia Developer's Community,

  11. #11
    Registered User
    Join Date
    Apr 2010
    Posts
    15

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Quote Originally Posted by raj_J2ME View Post
    That depends upon the situation.
    What do you mean by the multiple request?
    Why dont you put the request on some event like keypressed() or command?

    Thanks for your quick reply! I have a thread that periodically performs HTTP requests. It opens a connection, sends some data and closes it. This is repeated many times. When it reaches a certain limit i get the OutOfMemoryError, even though I seem to release all resources after every request.

  12. #12
    Super Contributor
    Join Date
    Mar 2008
    Location
    The Capital of INDIA
    Posts
    4,328

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Quote Originally Posted by heldal View Post
    Thanks for your quick reply! I have a thread that periodically performs HTTP requests. It opens a connection, sends some data and closes it. This is repeated many times. When it reaches a certain limit i get the OutOfMemoryError, even though I seem to release all resources after every request.
    while (true) {
    try {
    i++;
    System.out.println("Request number: " + i);
    connection = (HttpConnection)Connector.open("http://skynet.tele2.no/services/addFootage/"); //openoutputstream?
    connection.setRequestMethod(HttpConnection.POST);
    connection.setRequestProperty("User-Agent", "SkyNet Mobile Client");
    connection.setRequestProperty("Content-Type", "application/octet-stream");
    connection.setRequestProperty("Content-Length", String.valueOf(sendData.length));
    outStream = connection.openOutputStream();
    outStream.write(sendData);
    outStream.flush();
    outStream.close();
    outStream = null;

    //perhaps read the response?
    connection.close();
    }
    catch (Exception e) {
    e.printStackTrace();
    }
    finally {
    Thread.yield();
    }

    }
    look at you while Loop, this running as quick as it can, And your requirement must be call the server after 1 minute or so on, so rather than putting the connection code inside the infinite loop, call this fix time interval basis.Don't forget to release all the resource when you have process the request.Surely this will fix the issue.
    Thanks with Regards,

    R a j - The K e r n e l


    Join Delhi-NCR Nokia Developer's Community,

  13. #13
    Registered User
    Join Date
    Apr 2010
    Posts
    15

    Re: The issue of "OutputStream.flush()"/Memory leak.

    Quote Originally Posted by raj_J2ME View Post
    look at you while Loop, this running as quick as it can, And your requirement must be call the server after 1 minute or so on, so rather than putting the connection code inside the infinite loop, call this fix time interval basis.Don't forget to release all the resource when you have process the request.Surely this will fix the issue.
    It doesn't matter if I wait a long time or not - the error occurs anyway. What do you mean by release all the resources? Don't I already do it in the code?

  14. #14
    Registered User
    Join Date
    Apr 2010
    Posts
    15

    Re: The issue of "OutputStream.flush()"/Memory leak.

    This code makes it last longer, however it still fails after a little while:

    Code:
    SocketConnection connection = null;
    		OutputStream outStream = null;
    		InputStream inStream = null;
    		try {
    			connection = (SocketConnection)Connector.open("socket://pling.no:80"); //openoutputstream?
    			connection.setSocketOption(SocketConnection.KEEPALIVE, 1);
    			outStream = connection.openOutputStream();
    			inStream = connection.openDataInputStream();
    		}
    		catch (Exception e) {
    			e.printStackTrace();
    		}
    		
    		byte[] sendData = new byte[1024];
    		byte[] retrieveData = new byte[1024];
    		int bytesReceived;
    		byte[] requestString = "GET / HTTP/1.1\nHost: pling.no\nConnection: Keep-Alive\n\n".getBytes();
    		int i = 0;
    		while (i < 10000) {
    				try {
    					i++;
    
    					System.out.println("Request number: " + i);
    
    					outStream.write(requestString);
    					//outStream.write(sendData);
    					outStream.flush();
    
    					 bytesReceived = inStream.read(retrieveData);
    					
    				}
    				catch (Exception e) {
    					e.printStackTrace();
    				}
    				finally {
    					try {
    						Thread.yield();
    						//Thread.sleep(20);
    					}
    					catch (Exception e) {e.printStackTrace();}
    				}
    				System.gc();
    
    
    		}
    Note that if I change Thread.yield() to Thread.sleep(20), I only get to perform around half the number of requests before the OutOfMemoryError appears. It also lasts a little longer if I skip outStream.flush(), but only a few percent.

    The memory monitor keeps flat, telling me I have tons of available memory.

Similar Threads

  1. Memory Leak Issue
    By davmt in forum Symbian
    Replies: 8
    Last Post: 2008-03-29, 18:00
  2. Memory Leak Issue
    By pavan in forum Symbian
    Replies: 5
    Last Post: 2006-11-28, 09:01
  3. memory leak on emulator = memory leak on device
    By manmli in forum Mobile Java General
    Replies: 2
    Last Post: 2006-08-05, 19:54
  4. Memory Leak issue
    By symsahoo in forum Symbian
    Replies: 1
    Last Post: 2006-07-28, 12:31
  5. Memory leak issue Nokia 6100
    By gadkii in forum Mobile Java General
    Replies: 0
    Last Post: 2004-08-14, 02:38

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×