Replacing QNetworkAccessManager for the great good

Everybody using Qt for networking for small tasks will sometimes face oddities of QNetworkAccessManager. This class aims to be useful and convenient while having few quite sensible drawbacks. First one of couse is inability to use it in blocking way. What you should do instead is to create instance of QEventLoop and connect it’s quit() signal with network manager.

QNetworkAccessManager networkManager;
QEventLoop loop;
QNetworkReply *netReply = networkManager.get(resource);
connect(netReply, SIGNAL(finished()), &loop, SLOT(quit()));

This is overkill and overengineering of course. This inconveniency strikes also when you try to use it from background thread for downloading something – QNetworkAccessManager needs an event loop and it will launch one more thread – it’s own to do all the operations required.

Also it has a lot of data, methods and abilities not needed for “everyday simple network operations” like querying some API or downloading files. I don’t know anybody who wasn’t looking for a substitude for it at least once. But fortunately the solution exists.


Resources to learn and understand parallel programming. The hard way

There’s no way other than the hard way. (c)

Parallel programming is considered as not easy or even advanced topic by many programmers. It’s the starting point for even more advanced stuff like distributed computations, reliability, CAP theorem, consensus problems and much more. Besides, deep understanding of how CPU and operating system works can help you to write less buggy software and parallel programming can help you with that too.

In this post I will focus on books describing parallel programming using 1 computer and 1 CPU using classical approaches. Neither they contain SSE instructions guides nor you will find matterials on CUDA or OpenCL. Similary you will find no resourced about Hadoop and/or MapReduce technologies and nothing about technologies supporting parallel programming out of the box like Go or Erlang.

So I will go now through all the resources which I find more or less useful. I’m not going to stick to any technology in general – the point is to understand the topic from different perspectives. The materials I’m refering to in general should not be considered as entry-level –  they require fair amount of knowledge, but nevertheless, list goes sorted starting from “easier” things.


Classic Producer-Consumer in Qt/C++

Producer-Consumer is a classic pattern of interaction between two or more threads which share common tasks queue and workers who process that queue. When I came to similar task first I googled for standard approaches in Qt to solve this problem, but they were based on signals/slots plus synchronization primitives while I wanted simple and clear solution. Of course, in the end I’ve invented my own wheel and I invite you to take a look at it.

For the synchronization in Producer-Consumer it’s useful to use Mutex and some kind of WaitingEvent for synchronous waiting until mutex is acquired. In Qt you have QMutex and QWaitCondition which are all that we need.

Let’s suppose we have following data structures:

        QWaitCondition m_WaitAnyItem;
        QMutex m_QueueMutex;
        QVector<T*> m_Queue;

where T is type of messages we’re producing/consuming. So we have queue of elements being processed, mutex to secure access to the queue and wait condition to wait if the queue is empty.

For Producer-Consumer usually we need methods produce() and consume(). Let’s see how we can implement them.


Implementing autocomplete for English in C++

When it comes to implementing autocompletion in C++ in some type of input field, the question is which algorithm to choose and where to get the source for completion. In this post I’ll try to answer both questions.

As for the algorithm, SO gives us hints about tries, segment trees and others. You can find good article about them. Author has implemented some of them in a repository called FACE (fastest auto-complete in the east). You can easily find it on GitHub. This solution is used for the autocompletion in search engine Duck-Duck-Go which should tell you how good it is. Unfortunately their solution requires dependencies on libuv and joyent http-parser, which is not good in case you need just to integrate autocompletion functionality into your C++ application, but not build auto-complete server and send queries to it. Another drawback – libuv and cpp-libface itself fails to compile in Windows which is bad in case you’re building cross-platform solution.

You can find out how to built FACE into your cross-platform C++ application below.


Tips and tricks to improve performance of your ACM solution

Here I gathered system-programming tricks that can improve performance of your solution in C++ dramatically!

  • Use scanf/printf functions for standard IO instead of cin/cout
  • Memory-align buffers and structures to WORD size of your architecture (4 bytes for 32-bit and 8 bytes for 64-bit)
  • Use arrays instead of linked lists (to use memory block caching)
  • Avoid “if” stamements in loops
  • If-close should contain code, which is more likely to execute (if-condition == true)
  • Use inlining for short functions
  • Use objects allocated on stack but not on heap (local objects for functions instead of allocated with malloc/new)
  • Use pre-calculated hardcoded data (e.g. you can store first N prime numbers or first N Fibonacci numbers in order not to calculate them every time you need one)

Interesting issues and features of Qt programming

In this post I enlist all interesting facts and issues I’ve experienced while developing my first project in Qt (from 5.3 to 5.4 5.6)

18.02 – QSettings interface in Qml transformed bool to string and was always true on deserialization

18.02 – QByteArray returned from local scope crashed with heap corruption on return of function (destructor of QByteArray)

Qt can delete your object in it’s gui loop. QObject should have CppManaged attribute and it should be set before returning object to UI code.

26.02 QTimer can be only started when EventLoop is already running (e.g. with app.exec()) otherwise you get an error “QTimer can be started in QThread”

Drag and Drop files in OS X inserts NSUrl instead of QUrl and you have to convert it using Objective C to real filepath. You can add .m file to Qt project and write C++/Objective-C code.

23.03 QtConcurrent::mapped can accept a struct with operator() but only with inner typedef T return_type; where T is mapped type for correct QFuture<T> conversion. Also, QtConcurrent::mapped cannot be cancelled.

30.05 Qt Column allows to do animations, but has issues with stretching and ColumnLayout has no issues with stretching, but doesn’t allow you to do animations. I had to use simple anchors layout and States with Transitions to animate properties I wanted

1.06 Qt ListView has Transitions add/remove/removeDisplaced/displaced/etc, which allow to create nice animations for adding/deleting items from listview and to make UI really nice

6.06 Qt lacks standard Zip/Unzip functionality. QuaZip makes life easier, but has minor issues with compilation (linking etc).

22.07 TextInput’s EditingFinished signal fires twice in OS X

6.09 Windeployqt does not pack qml dlls into the bundle, you have to do it manually from your <Qt-bin-dir>/qml. You need to copy QtQuick and QtQuick2 directories and others which you use

19.09 Tab control is a loader and you can’t access it’s child object by Id. You have to create a property of Tab and do double binding inside and outside Tab to that property.

25.10 Repeater in GridLayout does not respect QAbstractItemModel changes, but does respect once put into Flow

23.11 Always initialize boolean fields, volatile or not. Always do that because in other case you will get tons of unpredictable behavior.

28.11 Use QVector instead of QList for most types. If you have released public version with wrong type and users depend on it, you’re screwed

12.12 QML ListView fails to update itself after sophisticated filter/remove operations. You have to manually positionViewAtBeginning() stuff

20.03 QFile in Windows does not respect flag QIODevice::Unbuffered because of “lack of native support in Windows” (bullshit) as of Qt 5.6

21.03 QProcess has super strange problems. Exiftool does not work with unicode from QProcess but works when launched via cmd.exe

2.05 QWaitCondition destructor produced a warning that it’s destroyed while inner Mutex is still locked. Was able to debug that using export QT_FATAL_WARNINGS=true in the running environment

20.07 QString::fromLatin1() truncated buffer of QByteArray parameter if the latter contained some eol/nl characters

16.12 virtual inheritance from the QObject is not supported.. Need to have stubs for calling signals

17.03 Timer in QML can have very high CPU utilization. Need to use with care

Text encryption in Qt/C++ with tiny AES 128bit

Have you ever needed a small, really small encryption in your C++ project for some piece of text? Say, credentials, login details or any other sensitive data? Of course, the best way is to keep just hash of salted password, but… What if you just need to do it and the size is so much critical for you?

There’re openSSL library and Crypto++ library which are monsters with tons of encryption algorithms, used in a number of solid projects etc. But.. they are big! I don’t want 30Mb library in my tiny project, which weights 10 Mb with high-resolution icons for OS X which weight by itself 5Mb. So I don’t want to sacrifice the size but still need encryption. Meet tiny-AES. It’s really small AES 128-bit library which does encryption in CBC and ECB modes. It really contains everything you needed just to encrypt and decrypt your sensitive data and forget about it.

You can find example under the hood.


.NET String Dictionary vs string switch performance

I had a simple task to map a collection of objects with string property. Map function should replace one string property to another from set of 5-6 strings. Existing solution used Dictionary initialized with those hard-coded values. Once upon a time I tried to compare Dictionary with int keys to int switch and int switch was FAR better. It was chess engine so performance mattered.

Now it’s a web request with thousands of rows of reply, serialized to json, so performance matters again. I wrote simple program, which generated 10 million instances of my simple class with several properties and mapped this list with both methods. Before that I ensured that both methods were JIT’ed. You can find source code at Github.

Details below..


Queem – open source computer chess in C#

I had an interview long time ago when I was young and just started programming. I had been showing my badly written battleship game to a skilled guy in his office. It was written in Delphi but with Pascal one-big-function style and it wasn’t double buffered so it has known rendering problems. I said then “You see, it is not a chess, but..”.

In a few weeks I started writing my own chess. After 2 years they were rewritten 3 times and now they are implemented in C# using bitboards (chessboard in 64-bit integer).
There’re some tests for AI engine, graphics in WPF, svg’s for chess figures and a lot more.

Checkout at a GitHub page!

I have found a lot of interesting materials while writing Queem which have one leader – chessprogramming wikipedia. It contains everything you want to begin write your own chess.

There is also nice web page about Rebel, great chess program.

And a list of materials about chess programming.