×

Discussion Board

Results 1 to 10 of 10
  1. #1
    Registered User
    Join Date
    Nov 2009
    Posts
    7

    XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    I am trying to receive a xml over HTTP and use CParser to parse it.

    The HTTP API has MHTTPDataSupplier which returns a dataChunk which is a HBufC which is a 16 bit representation, But the CParser APIs need a TDesc8 and am having a problem with this inconsistency. If i try to just take the data chunk as a TptrC8 and create a HBufC8 from that, I am losing every alternate character as shown in the sample below.

    Before conversion ::
    <?xml version="1.0" encoding="utf-8"?><content><userid>578283</userid></content>

    After Conversion ::
    <xlvrin"."ecdn=uf8?<otn>uei>723/srd<cnet<

    What is happenning is that the alternate characters are being lost in the 16 bit to 8 bit conversion!!!

    can anybody help me on this,

    Thanks in advance.

    Here i am attaching my code pieces.


    case THTTPEvent::EGotResponseBodyData:
    {
    // Get the body data supplier
    MHTTPDataSupplier* body = aTransaction.Response().Body();


    TPtrC8 dataChunk;

    // GetNextDataPart() returns ETrue, if the received part is the last
    // one.
    TBool isLast = body->GetNextDataPart(dataChunk);

    RDebug::Print( _L("***** ResponseBuffer length: %d"), dataChunk.Length() );



    HBufC* buf = HBufC::NewLC(dataChunk.Length());
    buf->Des().Copy(dataChunk);
    RDebug::Print( _L("***** buf: %S"), buf );


    // Append to iResponseBuffer
    if (iResponseBuffer==NULL)
    {
    iResponseBuffer = buf->AllocL();
    RDebug::Print( _L("***** ResponseBuffer: %S"), iResponseBuffer );
    }
    else
    {
    RDebug::Print( _L("***** ResponseBuffer content length: %d"), iResponseBuffer->Length()+buf->Length() );

    iResponseBuffer = iResponseBuffer->ReAllocL(iResponseBuffer->Length()+buf->Length());
    iResponseBuffer->Des().Append(*buf);
    }
    // xml parser StartParsingL method with TDesC

    iXmlHandler->StartParsingL(*iResponseBuffer);

    ------------------
    XmlHandler class
    ------------------
    void CXmlHandler::StartParsingL( TDesC& xmlContent )
    {


    HBufC8 *pHeapUTF = CnvUtfConverter::ConvertFromUnicodeToUtf8L(xmlContent);
    RDebug::Print( _L("***** content: %S"), pHeapUTF );

    iParser->ParseBeginL();
    iParser->ParseL( *pHeapUTF);
    iParser->ParseEndL();


    }


    I tried with the following ::


    HBufC* buf = HBufC::NewLC(xmlContent.Length());
    buf->Des().Copy(xmlContent);


    TPtrC8 representation((TUint8*)buf->Ptr(), buf->Size());

    HBufC8* myBuf = HBufC8::NewLC( buf->Size());
    myBuf->Des().Copy(representation);
    RDebug::Print( _L("***** content8: %S"), myBuf );


    Getting null characters at the end and missing even characters also.

    Log information ::



    26.975 ***** ResponseBuffer length: 80
    26.975 ***** buf: <?xml version="1.0" encoding="utf-8"?><content><userid>578283</userid></content>
    26.975 ***** ResponseBuffer: <?xml version="1.0" encoding="utf-8"?><content><userid>578283</userid></content>
    26.985 ***** ResponseBuffer: <?xml version="1.0" encoding="utf-8"?><content><userid>578283</userid></content>
    27.000 ***** content: <xlvrin"."ecdn=uf8?<otn>uei>723/srd<cnet<

  2. #2
    Registered User
    Join Date
    Jan 2007
    Posts
    282

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    I guess the problem lies here. You arent performing unicode conversion of incoming TDesC8 data.
    Code:
    HBufC* buf = HBufC::NewLC(dataChunk.Length());
    buf->Des().Copy(dataChunk);
    RDebug::Print( _L("***** buf: %S"), buf );
    Changing that to
    HBufC* buf = CnvUtfConverter::ConvertToUnicodeFromUtf8L(dataChunk)

    should solve the issue.

    Also one more point. Why are your trying to convert TDesC8 data to Unicode and then back to TDesC8?

    Hope that helps,
    VJ

  3. #3
    Regular Contributor
    Join Date
    Dec 2009
    Posts
    77

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    If you using only english chars in xml, Try this

    void CXmlHandler::StartParsingL( TDesC& xmlContent )
    {
    HBufC8 *pHeap = HBufC8::NewL(xmlContent.Length());
    TPtr8 pHeapPtr(pHeap->Des());
    pHeapPtr.Copy(xmlContent);
    RDebug::Print( _L("***** content: %S"), pHeapPtr );

    iParser->ParseBeginL();
    iParser->ParseL( pHeapPtr);
    iParser->ParseEndL();
    }

  4. #4
    Registered User
    Join Date
    Nov 2009
    Posts
    7

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    Hi Vaibhav,
    The problem is that the data that I am receiving in the datachunk is expected to be a xml string as described in the sample data.

    But when we alloc that buffer into a HBufC8 to send it to the CParser, the alternate characters are lost!!!

    So <?xml .... in the data chunk, is displayed as <xl ... So my parser is failing.

    but when we alloc from the datachunk to a HBufC then the <?xml is printed correctly as <?xml , but I am not able to send this to the CParser methods since they expect a TDesC8

    thanks,


    Quote Originally Posted by vaibhavjain View Post
    I guess the problem lies here. You arent performing unicode conversion of incoming TDesC8 data.
    Code:
    HBufC* buf = HBufC::NewLC(dataChunk.Length());
    buf->Des().Copy(dataChunk);
    RDebug::Print( _L("***** buf: %S"), buf );
    Changing that to
    HBufC* buf = CnvUtfConverter::ConvertToUnicodeFromUtf8L(dataChunk)

    should solve the issue.

    Also one more point. Why are your trying to convert TDesC8 data to Unicode and then back to TDesC8?

    Hope that helps,
    VJ

  5. #5
    Registered User
    Join Date
    Jan 2007
    Posts
    282

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    You are loosing alternate charecter due to inconsistant conversions that you are performing. Form converting TDesC8 to TDesC you are uing TDesC::Copy method as in

    Code:
    buf->Des().Copy(dataChunk);
    but for conversion from TDesC8 to TDesC you are using

    Code:
    HBufC8 *pHeapUTF = CnvUtfConverter::ConvertFromUnicodeToUtf8L(xmlContent);
    ;

    CnvUtfConverter::ConvertFromUnicodeToUtf8L will simply strip each alternate byte from the input (hi byte of the unicode charecter) and return the remaining bytes. Thats why you are loosing half of your data. Since xml is textual data (unicode) the best way to convert it TO/FROM TDesC8(UFT) is to use CnvUtfConverter class's static methods.

    Alternate method would be (if you are only using UTF-8 data) is to use the TDes16/8::Copy() methods. But whatever methods you choose be sure to use same for both way conversions.

    Your bug is simply due to mixing up of two conversion procedures. 2 Posts above ( 1 from me and another from qt_phantom) provide valid solutions to this bug.

    Hope that helps,
    VJ

  6. #6
    Registered User
    Join Date
    Nov 2009
    Posts
    7

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    Here i am using the same conversions from CnvUtfConverter

    TBool isLast = body->GetNextDataPart(dataChunk);
    RDebug::Print( _L("***** ResponseBuffer length: %d"), dataChunk.Length() );
    HBufC* unicode = CnvUtfConverter::ConvertToUnicodeFromUtf8L(dataChunk);
    HBufC8* utf = CnvUtfConverter::ConvertFromUnicodeToUtf8L(*unicode);

    RDebug::Print( _L("***** unicode: %S"), unicode );
    RDebug::Print( _L("***** utf: %S"), utf );

    Log is ::

    34.430 ***** ResponseBuffer length: 80
    34.435 ***** unicode: <?xml version="1.0" encoding="utf-8"?><content><userid>578283</userid></content>
    34.440 ***** utf: <xlvrin"."ecdn=uf8?<otn>uei>723/srd<cnet

  7. #7
    Regular Contributor
    Join Date
    Dec 2009
    Posts
    77

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    did you tried copy a 16-bit discripter directly to 8-bit discripor as in my previous reply? without using CnvUtfConverter etc.

  8. #8
    Registered User
    Join Date
    Nov 2009
    Posts
    7

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    Yes i tried with

    HBufC8 *pHeap = HBufC8::NewL(xmlContent.Length());
    TPtr8 pHeapPtr(pHeap->Des());
    pHeapPtr.Copy(xmlContent);
    RDebug::Print( _L("***** content: %S"), pHeapPtr );

    getting User Panic 7

  9. #9
    Super Contributor
    Join Date
    Mar 2004
    Location
    Singapore
    Posts
    9,968

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    RDebug::Print( _L("***** content: %S"), pHeapPtr ); has problem.

  10. #10
    Regular Contributor
    Join Date
    Dec 2009
    Posts
    77

    Re: XML Response Parsing Problem with CParser, TDesC to TDesC8 Conversion problem

    sorry,
    yes problem is
    RDebug::Print( _L("***** content: %S"), pHeapPtr );

    it should be

    RDebug::Print( _L("***** content: %S"), pHeap );

Similar Threads

  1. Problem retreiving custom HTTP response header
    By hmeet2 in forum Symbian Networking & Messaging (Closed)
    Replies: 0
    Last Post: 2007-01-03, 09:35
  2. problem in parsing xml file using syexpat
    By sunny_singh in forum Symbian
    Replies: 21
    Last Post: 2006-07-17, 11:56
  3. XML parsing problem
    By tugmbh in forum Mobile Java General
    Replies: 1
    Last Post: 2006-06-17, 15:41
  4. XML parsing problem in Nokia 12
    By codekiller in forum Nokia M2M
    Replies: 2
    Last Post: 2005-04-26, 09:01
  5. nokia 7210 modem via IR doesn't work !
    By oussamaaiadi in forum PC Suite API and PC Connectivity SDK
    Replies: 1
    Last Post: 2003-03-06, 10:46

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
×