×

Discussion Board

Results 1 to 7 of 7
  1. #1
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Download with HTTP crashes files - how to fix it?

    Hi,
    I try to download some files, mostly archives (*.7z, *.rar), using HTTP.
    My code for it is:
    Code:
    void downloader::Do_Download()
    {
        http = new QHttp(this);
    
        connect(http, SIGNAL(stateChanged(int)), this, SLOT(stateChanged(int)));
        connect(http, SIGNAL(responseHeaderReceived(QHttpResponseHeader)), this, SLOT(responseHeaderReceived(QHttpResponseHeader)));
        connect(http, SIGNAL(requestFinished(int,bool)), this, SLOT(requestFinished(int,bool)));
    
        QString host = "***"; //
        qDebug() << "Host: " + host;
    
        QString dl = "***";
        qDebug() << "Datei: " + dl;
    
    
        http->setHost(host);
        http->get(dl);
    }
    
    void downloader::stateChanged ( int state )
    {
        switch (state)
        {
            case 0:
                qDebug() << "Unconnected";
                break;
            case 1:
                qDebug() << "Hhost Lookup";
                break;
            case 2:
                qDebug() << "Connecting";
                break;
            case 3:
                qDebug() << "Sending";
                break;
            case 4:
                qDebug() << "Reading";
                break;
            case 5:
                qDebug() << "Connect";
                break;
            case 6:
                qDebug() << "Close";
                break;
        }
    }
    
    void downloader::responseHeaderReceived ( const QHttpResponseHeader & resp )
    {
        qDebug() << "Size" << resp.contentLength();
        qDebug() << "Type" << resp.contentType();
        qDebug() << "Status" << resp.statusCode();
        qDebug() << "Content" << resp.toString();
    
        QFile file("C:/html-test/resp.txt");
        if (file.open(QIODevice::WriteOnly))
        {
            QTextStream stream(&file);
            stream << resp.toString();
            file.close();
        }
    }
    
    void downloader::requestFinished ( int id, bool error )
    {
        if (error)
        {
            qDebug() << "ERROR";
        }
        else
        {
            qDebug() << "OK";
    
            QFile file("C:/html-test/test.rar");
            if (file.open(QIODevice::WriteOnly))
            {
                QTextStream stream(&file);
                stream << http->readAll();
                file.close();
            }
        }
    }
    It does download the archives, but it crashes the files. WinRAR reports that the file headers are corrupt and 7Zip reports that the hole archive is corrupt.

    So I downloaded some archives using download managers contained in Webbrowsers like Firefox and Opera. I compared my "corrupted" archives with them from the browsers and found out:
    The archives are really corrupt, there are differences at some bytes...

    The questions are:
    1. Is it at server problem (do not think so)?
    2. Is it impossible to use this code to download archives (seems to work with "text" files like .txt or .php)?
    3. If it is possible: What do I have to change to get it working?


    Maybe these are dumb questions but downloads are a absolutely new topic for me.
    Last edited by _DICE_; 2011-12-30 at 11:09.

  2. #2
    Registered User
    Join Date
    Jul 2004
    Posts
    166

    Re: Download with HTTP crashes files - how to fix it?

    I had created download manager with QNetworkAccessManager. Here is link

    http://kunalmaemo.blogspot.com/2011/...ith-pause.html

    you can try this code and check if it works. And one more thing to write binary data try to use QDataStream instead of QTextStrem class.
    http://kunalmaemo.blogspot.com/

  3. #3
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Re: Download with HTTP crashes files - how to fix it?

    Thank you, the hint with the QDataStream class helped - it works now.

  4. #4
    Registered User
    Join Date
    Jul 2004
    Posts
    166

    Re: Download with HTTP crashes files - how to fix it?

    I was suspecting QTextStream for cause, Glad that you solved your problem.
    http://kunalmaemo.blogspot.com/

  5. #5
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Re: Download with HTTP crashes files - how to fix it?

    Again I have a problem with my downloads.
    I moved the downloads in a separate thread and the result is:
    Quote Originally Posted by debug log
    "the url I entered"
    "host extracted from the url"
    "file extracted from the url"
    Fertig -> fired by requestFinished
    Connecting
    Sending
    Reading
    Size 0
    Type "text/html"
    Status 302
    Inhalt "HTTP/1.1 302 Moved Temporarily
    Date: Sat, 07 Jan 2012 19:42:21 GMT
    Server: Apache
    Set-Cookie: 00016ec1e4295b42ce874cf12ffd17b1=-; path=/
    Expires: Mon, 26 Jul 1997 05:00:00 GMT
    Last-Modified: Sat, 07 Jan 2012 19:42:21 GMT
    Cache-Control: no-store, no-cache, must-revalidate
    Cache-Control: post-check=0, pre-check=0
    Pragma: no-cache
    Location: here I get a new url
    Vary: Accept-Encoding
    Keep-Alive: timeout=15, max=100
    Connection: Keep-Alive
    Transfer-Encoding: chunked
    Content-Type: text/html

    "
    "neue url: url extracted from response
    "
    Connected
    Fertig -> ok we're at the end, so this may be correct
    Close
    Unconnected
    downloadthread.h
    Code:
    #ifndef DOWNLOADTHREAD_H
    #define DOWNLOADTHREAD_H
    
    #include <QThread>
    #include <QHttp>
    
    class DownloadThread : public QThread
    {
        Q_OBJECT
    public:
        explicit DownloadThread(QObject *parent = 0);
    
        void run();
    
        QString url;
    
        int step;
    
    signals:
    
    public slots:
        void stateChanged(int state);
        void responseHeaderReceived(const QHttpResponseHeader &resp);
        void dataReadProgress(int read, int total);
        void requestFinished(int id, bool error);
    
    private:
        QHttp *http;
    
    private slots:
        void Do_Download();
    
    };
    
    #endif // DOWNLOADTHREAD_H
    downloadthread.cpp
    Code:
    #include "downloadthread.h"
    
    #include <QDebug>
    #include <QHttp>
    #include <QFile>
    #include <QApplication>
    
    DownloadThread::DownloadThread(QObject *parent) :
        QThread(parent)
    {
    }
    
    void DownloadThread::run()
    {
        Do_Download();
    }
    
    void DownloadThread::Do_Download()
    {
        qDebug() << url;
    
        if (url.startsWith("http://"))
        {
            url.remove("http://");
        }
    
        http = new QHttp();
        http->moveToThread(QApplication::instance()->thread());
        connect(http, SIGNAL(stateChanged(int)), this, SLOT(stateChanged(int)));
        connect(http, SIGNAL(responseHeaderReceived(QHttpResponseHeader)), this, SLOT(responseHeaderReceived(QHttpResponseHeader)));
        connect(http, SIGNAL(dataReadProgress(int,int)), this, SLOT(dataReadProgress(int, int)));
        connect(http, SIGNAL(requestFinished(int,bool)), this, SLOT(requestFinished(int,bool)));
    
        QString host = url.mid(0, url.indexOf("/"));
        QString dl = url.mid(url.indexOf("/"));
    
        qDebug() << host;
        qDebug() << dl;
    
        http->setHost(host);
        http->get(dl);
    }
    
    void DownloadThread::stateChanged(int state)
    {
        switch (state)
        {
            case 0:
                qDebug() << "Unconnected";
                break;
            case 1:
                qDebug() << "Host Lookup";
                break;
            case 2:
                qDebug() << "Connecting";
                break;
            case 3:
                qDebug() << "Sending";
                break;
            case 4:
                qDebug() << "Reading";
                break;
            case 5:
                qDebug() << "Connected";
                break;
            case 6:
                qDebug() << "Close";
                this->terminate();
                break;
        }
    }
    
    void DownloadThread::responseHeaderReceived(const QHttpResponseHeader &resp)
    {
        qDebug() << "Size" << resp.contentLength();
        qDebug() << "Type" << resp.contentType();
        qDebug() << "Status" << resp.statusCode();
        qDebug() << "Inhalt" << resp.toString();
    
        if (resp.toString().contains("Location: "))
        {
            QStringList splitted1 = resp.toString().split("Location: ");
            QStringList splitted2 = splitted1.at(1).split("Vary");
    
            url = splitted2.at(0);
            qDebug() << "neue url: " + url;
        }
    }
    
    void DownloadThread::dataReadProgress(int read, int total)
    {
        //qDebug() << read + "/" + total;
    }
    
    void DownloadThread::requestFinished(int id, bool error)
    {
        if (error)
        {
            qDebug() << "Fehler";
        }
        else
        {
            qDebug() << "Fertig";
            QFile file(QApplication::applicationDirPath() + "/test.7z");
            if (file.open(QIODevice::WriteOnly))
            {
                QTextStream stream(&file);
                stream << http->readAll();
                file.close();
            }
    
        }
    }
    My are questions are now: Why is requestFinished fired before it has connected to the host? How can I fix this?

    And my problem:
    I have to send 3 requests. The first contains a url you can find at a website (request 1).
    The response header gives me another url which I have to send (request 2). Again I get another url in the response header. This is the "real" url.
    Now I have to request the "real" url, which allows me to download the file.

    This should not be a problem. Normally I would call do it like this:
    1. set url-var with thread->url = "bla"
    2. start thread
    3. call Do_Download from "case 6" of stateChanged because here I should have a new url


    But this is the debug log:
    "url I set myself"
    "host extracted from url"
    "file extracted from url"
    Fertig -> Again that false one
    Connecting
    Sending
    Reading
    Size 0
    Type "text/html"
    Status 302
    Inhalt "HTTP/1.1 302 Moved Temporarily
    Date: Sat, 07 Jan 2012 19:51:12 GMT
    Server: Apache
    Set-Cookie: 00016ec1e4295b42ce874cf12ffd17b1=-; path=/
    Expires: Mon, 26 Jul 1997 05:00:00 GMT
    Last-Modified: Sat, 07 Jan 2012 19:51:12 GMT
    Cache-Control: no-store, no-cache, must-revalidate
    Cache-Control: post-check=0, pre-check=0
    Pragma: no-cache
    Location: here I get the new url
    Vary: Accept-Encoding
    Keep-Alive: timeout=15, max=100
    Connection: Keep-Alive
    Transfer-Encoding: chunked
    Content-Type: text/html

    "
    "neue url: my new url from response header (and it seems to be correct
    "
    Connected
    Fertig
    Close

    (request 2 starts here)
    "the url I got from response header"
    "host extraction works again"
    "file extraction works again too"
    Fertig -> *ugh*
    Connecting
    Unconnected
    Sending
    Reading
    Close -> What happend here? I did not get a response...
    I hope someone can help me with my two little problems because I tried to find the solution for a hole week now.
    Of course I could start the thread 3 times to get the urls and the files, but I think this is not a real solution.

  6. #6
    Registered User
    Join Date
    Jul 2004
    Posts
    166

    Re: Download with HTTP crashes files - how to fix it?

    have you checked my download manager implementation. That might give you some hints.
    http://kunalmaemo.blogspot.com/

  7. #7
    Registered User
    Join Date
    Jun 2011
    Posts
    12

    Re: Download with HTTP crashes files - how to fix it?

    It seems to be, more or less, the same. The only difference is you use Networkmanager and you have a pause/resume function. I will try to rewrite my download function with Networkmanager instead of Http.

    But I wonder why my function does not work anymore. It worked while I called it directly from the the gui thread. Now, after moving it in a separate thread, I get these errors. So I think it has something to do with the threads and I want to know what exactly the problems because I do not like it if something does not work and I do not know why.

Similar Threads

  1. Download multiple files using http
    By kuddy_abc in forum Symbian Networking & Messaging (Closed)
    Replies: 4
    Last Post: 2010-05-28, 05:08
  2. Replies: 3
    Last Post: 2009-04-09, 15:40
  3. Replies: 0
    Last Post: 2003-07-24, 10:46
  4. Upload and Download files using Wap instead of HTTP
    By arizkohli in forum Symbian Networking & Messaging (Closed)
    Replies: 0
    Last Post: 2003-07-19, 22:19

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •