In this blog post, I am going to review the different kind of data and how they are initialized in a program.

What I am going to explain here is valid for Linux and GCC.

Code Example

I'll just start by showing a small piece of code. What is going to interest us is where the data will end up in memory and how it is initialized.

const char string_data[] = "hello world"; // .rodata
const int even_numbers[] = { 0*2 , 1*2,  2*2,  3*2, 4*2}; //.rodata

int all_numbers[] = { 0, 1, 2, 3, 4 };  //.data

static inline int odd(int n) { return n*2 + 1; }
const int odd_numbers[] = { odd(0), odd(1), odd(2), odd(3), odd(4) }; //initialized

QString qstring_data("hello QString"); //object with constructor and destructor

I'll analyze the assembly. It has been generated with the following command, then re-formatted for better presentation in this blog post.

g++ -O2 -S data.cpp

(I also had to add a function that uses the data in order to avoid that the compiler removes some arrays that were not used.)

The sections

On Linux, the binaries (program or libraries) are stored as file in the ELF format. Those files are composed of many sections. I'll just go over a few of them:

The code: .text

This section is the actual code of your library or program it contains all the instructions for each function. That part of the code is mapped into memory, and shared between the instances of the processes that uses it (provided the library is compiled as position independent, which is usually the case).

I am not interested in the code in this blog post, let us move to the data sections.

The read-only data: .rodata

This section will be loaded the same way as the .text section is loaded. It will also be shared between processes.

It contains the arrays that are marked as const such as string_data and even_numbers.

.section    .rodata
_ZL11string_data:
    .string "hello world"
_ZL12even_numbers:
    .long   0
    .long   2
    .long   4
    .long   6
    .long   8

You can see that even if the even_numbers array was initialized with multiplications, the compiler was able to optimize and generate the array at compile time.

The _ZL11 that is part of the name is the mangling because it is const.

Writable data: .data

The data section will contain the pre-initialized data that are not read-only.
This section is not shared between processes but copied for each instance of processes that uses it. (Actually, with the copy-on-write optimization in the kernel, it might need to be copied only if the data changes.)

There goes our all_number array that has not been declared as const.

.data
all_numbers:
    .long   0
    .long   1
    .long   2
    .long   3
    .long   4

Initialized at run-time: .bss + .ctors

The compiler was not able to optimize the calls to odd(), it has to be computed at run-time. Where will our odd_numbers array be stored?

What will happen is that it will not be stored in the binary, but some space will be reserved in the .bss section. That section is just some memory which is allocated to each process, it is initialized to 0.

The binary also contains a section with code that is going to be executed before main() is being called.

.section    .text.startup
_GLOBAL__sub_I_odd_numbers:
    movl    $1, _ZL11odd_numbers(%rip)
    movl    $3, _ZL11odd_numbers+4(%rip)
    movl    $5, _ZL11odd_numbers+8(%rip)
    movl    $7, _ZL11odd_numbers+12(%rip)
    movl    $9, _ZL11odd_numbers+16(%rip)
    ret

.section    .ctors,"aw",@progbits
    .quad   _GLOBAL__sub_I_odd_numbers

.local  _ZL11odd_numbers  ; reserve 20 bytes in the .bss section
    .comm   _ZL11odd_numbers,20,16

The .ctor section contains a table of pointers to functions that are going to be called by the loader before it calls main(). In our case, there is only one, the code that initializes the odd_numbers array.

Global Object

How about our QString? It is a global C++ object with a constructor and destructor. It is simply initialized by running the constructor at start-up.

.section    .rodata.str1.1,"aMS",@progbits,1
.LC0:
    .string "hello QString"

.section    .text.startup,"ax",@progbits
_GLOBAL__sub_I_qstring_data:
       ; QString constructor (inlined)
    movl    $-1, %esi
    movl    $.LC0, %edi
    call    _ZN7QString16fromAscii_helperEPKci
    movq    %rax, _ZL12qstring_data(%rip)
       ; register the destructor
    movl    $__dso_handle, %edx
    movl    $_ZL12qstring_data, %esi
    movl    $_ZN7QStringD1Ev, %edi
    jmp __cxa_atexit   ; (tail call)

Here is the code of the constructor, which have been inlined.

We can also see that the code calls the function __cxa_atexit with the parameters $_ZL12qstring_data and $_ZN7QStringD1Ev Which are respectively the address of the QString object, and a function pointer of the QString destructor. In other words, this code registers the destructor of QString to be run on exit.
The third parameter $__dso_handle is a handle to this dynamic shared object (used to run the destructor when a plugin is unloaded for example).

What is the problem with global objects with constructor?

• The order in which the constructors are called are not specified by the C++ standard. If you have dependencies between your global objects, you will run into trouble.
• All the constructors of all the global in all the libraries need to be run before main() and slow down the startup of the application. (Even for objects that will never be used).

This is why it is not recommended to have global objects in libraries. Instead, one can use function static objects, which are initialized on the first use. (Qt provides a macro for that: Q_GLOBAL_STATIC which is made public in Qt 5.1.)

Here comes C++11

C++11 comes with a new feature: constexpr

That keyword can be used in two ways: If you specify that a function is a constexpr it means that the function can be run at compile-time.
If you specify that a variable is a constexpr, then it means it can be computed at compile time.

Let us slightly modify the example above and see what it does:

static inline constexpr int odd(int n) { return n*2 + 1; }
constexpr int odd_numbers[] = { odd(0), odd(1), odd(2), odd(3), odd(4) };

Two constexpr were added.

.section    .rodata
_ZL11odd_numbers:
    .long   1
    .long   3
    .long   5
    .long   7
    .long   9

Now they are generated at compile time.

If a class has a constructor that is declared as constexpr and has no destructor, you can have this as global object and it will be initialized at compile time.

Since Qt 4.8, there is a macro Q_DECL_CONSTEXPR which expands to constexpr if the compiler supports it, or to nothing otherwise.

QtQuick and QML form a really nice language to develop user interfaces. The QML Bindings are very productive and convenient. The declarative syntax is really a pleasure to work with.
Would it be possible to do the same in C++? In this blog post, I will show a working implementation of property bindings in pure C++.

Disclaimer: This was done for the fun of it and is not made for production.

Bindings

The goal of bindings is to have one property which depends on other properties. When its dependencies are changed, the property is automatically updated.

Here is an example inspired from the QML documentation.

int calculateArea(int width, int height) {
  return (width * height) * 0.5;
}

struct rectangle {
  property<rectangle*> parent = nullptr;
  property<int> width = 150;
  property<int> height = 75;
  property<int> area = [&]{ return calculateArea(width, height); };

  property<std::string> color = [&]{
    if (parent() && area > parent()->area)
      return std::string("blue");
    else
      return std::string("red");
  };
};

If you are not familiar with the [&]{ ... } syntax, this is a lambda function. I'm also using the fact that in C++11, you can initialize the members directly in the declaration.

Now, we'll see how this property class works. At the end I will show a cool demo of what you can do.

The code is using lots of C++11 constructs. It has been tested with GCC 4.7 and Clang 3.2.

Property

I have used my knowledge from QML and the QObject system to build something similar with C++ bindings.
The goal is to make a proof of concept. It is not optimized. I just wanted to have comprehensible code for this demo.

The idea behind the property class is the same as in QML. Each property keeps a list of its dependencies. When a binding is evaluated, all access to the property will be recorded as dependencies.

property<T> is a template class. The common part is put in a base class: property_base.

class property_base
{
  /* Set of properties which are subscribed to this one.
     When this property is changed, subscriptions are refreshed */
  std::unordered_set<property_base *> subscribers;

  /* Set of properties this property is depending on. */
  std::unordered_set<property_base *> dependencies;

public:
  virtual ~property_base()
  { clearSubscribers(); clearDependencies(); }

  // re-evaluate this property
  virtual void evaluate() = 0;
   
  // [...]
protected:
  /* This function is called by the derived class when the property has changed
     The default implementation re-evaluates all the property subscribed to this one. */
  virtual void notify() {
    auto copy = subscribers;
    for (property_base *p : copy) {
      p->evaluate();
    }
  }

  /* Derived class call this function whenever this property is accessed.
     It register the dependencies. */
  void accessed() {
    if (current && current != this) {
      subscribers.insert(current);
      current->dependencies.insert(this);
    }
  }

  void clearSubscribers() {
      for (property_base *p : subscribers)
          p->dependencies.erase(this);
      subscribers.clear();
  }
  void clearDependencies() {
      for (property_base *p : dependencies)
          p->subscribers.erase(this);
      dependencies.clear();
  }

  /* Helper class that is used on the stack to set the current property being evaluated */
  struct evaluation_scope {
    evaluation_scope(property_base *prop) : previous(current) {
      current = prop;
    }
    ~evaluation_scope() { current = previous; }
    property_base *previous;
  };
private:
  friend struct evaluation_scope;
  /* thread_local */ static property_base *current;
};

Then we have the implementation of the class property.

template <typename T>
struct property : property_base {
  typedef std::function<T()> binding_t;

  property() = default;
  property(const T &t) : value(t) {}
  property(const binding_t &b) : binding(b) { evaluate(); }

  void operator=(const T &t) {
      value = t;
      clearDependencies();
      notify();
  }
  void operator=(const binding_t &b) {
      binding = b;
      evaluate();
  }

  const T &get() const {
    const_cast<property*>(this)->accessed();
    return value;
  }

  //automatic conversions
  const T &operator()() const { return get();  }
  operator const T&() const { return get(); }

  void evaluate() override {
    if (binding) {
      clearDependencies();
      evaluation_scope scope(this);
      value = binding();
    }
    notify();
  }

protected:
  T value;
  binding_t binding;
};

property_hook

It is also desirable to be notified when a property is changed, so we can for example call update(). The property_hook class lets you specify a function which will be called when the property changes.

Qt bindings

Now that we have the property class, we can build everything on top of that. We could build for example a set of widgets and use those. I'm going to use Qt Widgets for that. If the QtQuick elements had a C++ API, I could have used those instead.

The property_qobject

I introduce a property_qobject which is basically wrapping a property in a QObject. You initialize it by passing a pointer to the QObject and the string of the property you want to track, and voilà.

The implementation is not efficient and it could be optimized by sharing the QObject rather than having one for each property. With Qt5 I could also connect to lambda instead of doing this hack, but I used Qt 4.8 here.

Wrappers

Then I create a wrapper around each class I'm going to use that expose the properties in a property_qobject

A Demo

Now let's see what we are capable of doing:

This small demo just has a line edit which lets you specify a color and few sliders to change the rotation and the opacity of a graphics item.

Let the code speak for itself.

We need a Rectangle object with the proper bindings:

struct GraphicsRectObject : QGraphicsWidget {
  // bind the QObject properties.
  property_qobject<QRectF> geometry { this, "geometry" };
  property_qobject<qreal> opacity { this, "opacity" };
  property_qobject<qreal> rotation { this, "rotation" };

  // add a color property, with a hook to update when it changes
  property_hook<QColor> color { [this]{ this->update(); } };
private:
  void paint(QPainter* painter, const QStyleOptionGraphicsItem* option, QWidget*) override {
    painter->setBrush(color());
    painter->drawRect(boundingRect());
  }
};

Then we can proceed and declare a window object with all the subwidgets:

struct MyWindow : Widget {
  LineEdit colorEdit {this};

  Slider rotationSlider {Qt::Horizontal, this};
  Slider opacitySlider {Qt::Horizontal, this};

  QGraphicsScene scene;
  GraphicsView view {&scene, this};
  GraphicsRectObject rectangle;

  ::property<int> margin {10};

  MyWindow() {
    // Layout the items.  Not really as good as real layouts, but it demonstrates bindings
    colorEdit.geometry = [&]{ return QRect(margin, margin,
                                             geometry().width() - 2*margin,
                                             colorEdit.sizeHint().height()); };
    rotationSlider.geometry = [&]{ return QRect(margin,
                                                  colorEdit.geometry().bottom() + margin,
                                                  geometry().width() - 2*margin,
                                                  rotationSlider.sizeHint().height()); };
    opacitySlider.geometry = [&]{ return QRect(margin,
                                                 rotationSlider.geometry().bottom() + margin,
                                                 geometry().width() - 2*margin,
                                                 opacitySlider.sizeHint().height()); };
    view.geometry = [&]{
        int x = opacitySlider.geometry().bottom() + margin;
        return QRect(margin, x, width() - 2*margin, geometry().height() - x - margin); 
    };

    // Some proper default value
    colorEdit.text = QString("blue");
    rotationSlider.minimum = -180;
    rotationSlider.maximum = 180;
    opacitySlider.minimum = 0;
    opacitySlider.maximum = 100;
    opacitySlider.value = 100;

    scene.addItem(&rectangle);

    // now the 'cool' bindings
    rectangle.color = [&]{ return QColor(colorEdit.text);  };
    rectangle.opacity = [&]{ return qreal(opacitySlider.value/100.); };
    rectangle.rotation = [&]{ return rotationSlider.value(); };
  }
};

int main(int argc, char **argv)
{
    QApplication app(argc,argv);
    MyWindow window;
    window.show();
    return app.exec();
}

Conclusion

You can clone the code repository and try it for yourself.

Perhaps one day, a library will provide such property bindings.

While working on my Qt developer days 2012 presentation (QtCore in depth), I made a benchmark comparing QMap and QHash. I thought it would be nice to share the results in this short blog entry.

Under The Hood

The Qt 4 containers are well explained by this old Qt Quarterly article.

QHash is implemented using a Hash Table and QMap was implemented using a Skip list in Qt4.

In Qt 5, the implementation of the containers have changed a bit, but the concepts are still the same. Here are the main differences:

• QVector, QString and QByteArray now share the same implementation ( QArrayData ). The main difference is that there is now an offset which might allow in the future to reference external data.
• QMap implementation has totally changed. It is no longer a skip list, but a red-black tree.

The Benchmark

The benchmark is simple and is doing lots of look-ups in a loop during one second and count the number of iterations.
It is not really scientific. The goal is only to show the shape of the curves.

The source: benchmark.cc

The Result

Run on my computer, gcc 4.7. Higher is better. The number of element is on a logarithmic scale. For QHash, one should expect it not to change with the number of elements, and for QMap it should be O(log N): a straight line on a logarithmic scale.

Qt 4.8

QMap performs slightly slower than std::map. QMap lookup is faster than in a QHash for less than about 10 elements.

Qt 5

It was a good idea to change from a skip list to a red-black tree. The performance of the Qt containers compared to the STL are about the same. QMap is faster than QHash if there is less than about 20 elements.

If you compare the number between Qt5 and Qt4 you see that Qt5 performs better. That might be related by the changes in QString.

Conclusion

The typical rule is: Use QMap only if you need the items to be sorted or if you know that you always have a very small amount of items in your map.

This post is about the use of QThread. It is an answer to a three years old blog post by Brad, my colleague at the time:
You're doing it wrong

In his blog post, Brad explains that he saw many users misusing QThread by sub-classing it, adding some slots to that subclass and doing something like this in the constructor:

 moveToThread(this);

They move a thread to itself. As Brad mentions, it is wrong: the QThread is supposed to be the interface to manage the thread. So it is supposed to be used from the creating thread.

Slots in the QThread object are then not run in that thread and having slots in a subclass of QThread is a bad practice.

But then Brad continues and discourages any sub-classing of QThread at all. He claims it is against proper object-oriented design. This is where I disagree. Putting code in run() is a valid object-oriented way to extend a QThread: A QThread represents a thread that just starts an event loop, a subclass represents a thread that is extended to do what's in run().

After Brad's post, some members of the community went on a crusade against sub-classing QThread. The problem is that there are many perfectly valid reasons to subclass QThread.

With Qt 5.0 and Qt 4.8.4, the documentation of QThread was changed so the sample code does not involve sub-classing. Look at the first code sample of the Qt 4.8 QThread documentation. It has many lines of boiler plate just to run some code in a thread. And the there is even a leak: the QThread is never going to quit and be destroyed.

I was asked on IRC a question from an user who followed that example in order to run some simple code in a thread. He had a hard time to figure out how to properly destroy the thread. That is what motivated me to write this blog entry.

If you allow to subclass QThread, this is what you got:

class WorkerThread : public QThread {
    void run() {
        // ...
    }
};

void MyObject::startWorkInAThread()
{
    WorkerThread *workerThread = new WorkerThread;
    connect(workerThread, SIGNAL(finished()),
            workerThread, SLOT(deleteLater()));
    workerThread->start();
}

This code does no longer leak and is much simpler and has less overhead as it does not create useless object.

The Qt threading example threadedfortuneserver is an example that uses this pattern to run blocking operations and is much simpler than the equivalent using a worker object.

I have submitted a patch to the documentation to not discourage sub-classing QThread anymore.

Rules of thumbs

When to subclass and when not to?

• If you do not really need an event loop in the thread, you should subclass.
• If you need an event loop and handle signals and slots within the thread, you may not need to subclass.

What about using QtConcurrent instead?

QThread is a quite low level and you should better use a higher level API such as QtConcurrent.

Now, QtConcurrent has its own set of problems: It is tied to a single thread pool so it is not a good solution if you want to run blocking operations. It has also some problems in its implementation that gives some performance overhead. All of this is fixable. Perhaps even Qt 5.1 will see some improvements.

A good alternative is also the C++11 standard library with std::thread and std::async which are now the standard way to run code in a thread. And the good news is that it still works fine with Qt: All other Qt threading primitives can be used with native threads. (Qt will create automatically create a QThread if required).

This is the sequel of my previous article explaining the implementation details of the signals and slots. In the Part 1, we have seen the general principle and how it works with the old syntax. In this blog post, we will see the implementation details behind the new function pointer based syntax in Qt5.

New Syntax in Qt5

The new syntax looks like this:

  QObject::connect(&a, &Counter::valueChanged,
                   &b, &Counter::setValue);

Why the new syntax?

I already explained the advantages of the new syntax in a dedicated blog entry. To summarize, the new syntax allows compile-time checking of the signals and slots. It also allows automatic conversion of the arguments if they do not have the same types. As a bonus, it enables the support for lambda expressions.

New overloads

There was only a few changes required to make that possible.
The main idea is to have new overloads to QObject::connect which take the pointers to functions as arguments instead of char*

There are three new static overloads of QObject::connect: (not actual code)

1. QObject::connect(const QObject *sender, PointerToMemberFunction signal,
                    const QObject *receiver, PointerToMemberFunction slot,
                    Qt::ConnectionType type)

2. QObject::connect(const QObject *sender, PointerToMemberFunction signal,
                    PointerToFunction method)

3. QObject::connect(const QObject *sender, PointerToMemberFunction signal,
                    Functor method)

The first one is the one that is much closer to the old syntax: you connect a signal from the sender to a slot in a receiver object. The two other overloads are connecting a signal to a static function or a functor object without a receiver.

They are very similar and we will only analyze the first one in this article.

Pointer to Member Functions

Before continuing my explanation, I would like to open a parenthesis to talk a bit about pointers to member functions.

Here is a simple sample code that declares a pointer to member function and calls it.

  void (QPoint::*myFunctionPtr)(int); // Declares myFunctionPtr as a pointer to
                                      // a member function returning void and
                                      // taking (int) as parameter
  myFunctionPtr = &QPoint::setX;
  QPoint p;
  QPoint *pp = &p;
  (p.*myFunctionPtr)(5); // calls p.setX(5);
  (pp->*myFunctionPtr)(5); // calls pp->setX(5);

Pointers to member and pointers to member functions are usually part of the subset of C++ that is not much used and thus lesser known.
The good news is that you still do not really need to know much about them to use Qt and its new syntax. All you need to remember is to put the & before the name of the signal in your connect call. But you will not need to cope with the ::*, .* or ->* cryptic operators.

These cryptic operators allow you to declare a pointer to a member or access it. The type of such pointers includes the return type, the class which owns the member, the types of each argument and the const-ness of the function.

You cannot really convert pointer to member functions to anything and in particular not to void* because they have a different sizeof.
If the function varies slightly in signature, you cannot convert from one to the other. For example, even converting from void (MyClass::*)(int) const to void (MyClass::*)(int) is not allowed. (You could do it with reinterpret_cast; but that would be an undefined behaviour if you call them, according to the standard)

Pointer to member functions are not just like normal function pointers. A normal function pointer is just a normal pointer the address where the code of that function lies. But pointer to member function need to store more information: member functions can be virtual and there is also an offset to apply to the hidden this in case of multiple inheritance.
sizeof of a pointer to a member function can even vary depending of the class. This is why we need to take special care when manipulating them.

Type Traits: QtPrivate::FunctionPointer

Let me introduce you to the QtPrivate::FunctionPointer type trait.
A trait is basically a helper class that gives meta data about a given type. Another example of trait in Qt is QTypeInfo.

What we will need to know in order to implement the new syntax is information about a function pointer.

The template<typename T> struct FunctionPointer will give us information about T via its member.

• ArgumentCount: An integer representing the number of arguments of the function.
• Object: Exists only for pointer to member function. It is a typedef to the class of which the function is a member.
• Arguments: Represents the list of argument. It is a typedef to a meta-programming list.
• call(T &function, QObject *receiver, void **args): A static function that will call the function, applying the given parameters.

Qt still supports C++98 compiler which means we unfortunately cannot require support for variadic templates. Therefore we had to specialize our trait function for each number of arguments. We have four kinds of specializationd: normal function pointer, pointer to member function, pointer to const member function and functors. For each kind, we need to specialize for each number of arguments. We support up to six arguments. We also made a specialization using variadic template so we support arbitrary number of arguments if the compiler supports variadic templates.

The implementation of FunctionPointer lies in qobjectdefs_impl.h.

QObject::connect

The implementation relies on a lot of template code. I am not going to explain all of it.

Here is the code of the first new overload from qobject.h:

template <typename Func1, typename Func2>
static inline QMetaObject::Connection connect(
    const typename QtPrivate::FunctionPointer<Func1>::Object *sender, Func1 signal,
    const typename QtPrivate::FunctionPointer<Func2>::Object *receiver, Func2 slot,
    Qt::ConnectionType type = Qt::AutoConnection)
{
  typedef QtPrivate::FunctionPointer<Func1> SignalType;
  typedef QtPrivate::FunctionPointer<Func2> SlotType;

  //compilation error if the arguments does not match.
  Q_STATIC_ASSERT_X(int(SignalType::ArgumentCount) >= int(SlotType::ArgumentCount),
                    "The slot requires more arguments than the signal provides.");
  Q_STATIC_ASSERT_X((QtPrivate::CheckCompatibleArguments<typename SignalType::Arguments,
                                                         typename SlotType::Arguments>::value),
                    "Signal and slot arguments are not compatible.");
  Q_STATIC_ASSERT_X((QtPrivate::AreArgumentsCompatible<typename SlotType::ReturnType,
                                                       typename SignalType::ReturnType>::value),
                    "Return type of the slot is not compatible with the return type of the signal.");

  const int *types;
  /* ... Skipped initialization of types, used for QueuedConnection ...*/

  QtPrivate::QSlotObjectBase *slotObj = new QtPrivate::QSlotObject<Func2,
        typename QtPrivate::List_Left<typename SignalType::Arguments, SlotType::ArgumentCount>::Value,
        typename SignalType::ReturnType>(slot);


  return connectImpl(sender, reinterpret_cast<void **>(&signal),
                     receiver, reinterpret_cast<void **>(&slot), slotObj,
                     type, types, &SignalType::Object::staticMetaObject);
}

You notice in the function signature that sender and receiver are not just QObject* as the documentation points out. They are pointers to typename FunctionPointer::Object instead. This uses SFINAE to make this overload only enabled for pointers to member functions because the Object only exists in FunctionPointer if the type is a pointer to member function.

We then start with a bunch of Q_STATIC_ASSERT. They should generate sensible compilation error messages when the user made a mistake. If the user did something wrong, it is important that he/she sees an error here and not in the soup of template code in the _impl.h files. We want to hide the underlying implementation from the user who should not need to care about it.
That means that if you ever you see a confusing error in the implementation details, it should be considered as a bug that should be reported.

We then allocate a QSlotObject that is going to be passed to connectImpl(). The QSlotObject is a wrapper around the slot that will help calling it. It also knows the type of the signal arguments so it can do the proper type conversion.
We use List_Left to only pass the same number as argument as the slot, which allows connecting a signal with many arguments to a slot with less arguments.

QObject::connectImpl is the private internal function that will perform the connection. It is similar to the original syntax, the difference is that instead of storing a method index in the QObjectPrivate::Connection structure, we store a pointer to the QSlotObjectBase.

The reason why we pass &slot as a void** is only to be able to compare it if the type is Qt::UniqueConnection.

We also pass the &signal as a void**. It is a pointer to the member function pointer. (Yes, a pointer to the pointer)

Signal Index

We need to make a relationship between the signal pointer and the signal index.
We use MOC for that. Yes, that means this new syntax is still using the MOC and that there are no plans to get rid of it :-).

MOC will generate code in qt_static_metacall that compares the parameter and returns the right index. connectImpl will call the qt_static_metacall function with the pointer to the function pointer.

void Counter::qt_static_metacall(QObject *_o, QMetaObject::Call _c, int _id, void **_a)
{
    if (_c == QMetaObject::InvokeMetaMethod) {
        /* .... skipped ....*/
    } else if (_c == QMetaObject::IndexOfMethod) {
        int *result = reinterpret_cast<int *>(_a[0]);
        void **func = reinterpret_cast<void **>(_a[1]);
        {
            typedef void (Counter::*_t)(int );
            if (*reinterpret_cast<_t *>(func) == static_cast<_t>(&Counter::valueChanged)) {
                *result = 0;
            }
        }
        {
            typedef QString (Counter::*_t)(const QString & );
            if (*reinterpret_cast<_t *>(func) == static_cast<_t>(&Counter::someOtherSignal)) {
                *result = 1;
            }
        }
        {
            typedef void (Counter::*_t)();
            if (*reinterpret_cast<_t *>(func) == static_cast<_t>(&Counter::anotherSignal)) {
                *result = 2;
            }
        }
    }
}

Once we have the signal index, we can proceed like in the other syntax.

The QSlotObjectBase

QSlotObjectBase is the object passed to connectImpl that represents the slot.

Before showing the real code, this is what QObject::QSlotObjectBase was in Qt5 alpha:

struct QSlotObjectBase {
    QAtomicInt ref;
    QSlotObjectBase() : ref(1) {}
    virtual ~QSlotObjectBase();
    virtual void call(QObject *receiver, void **a) = 0;
    virtual bool compare(void **) { return false; }
};

It is basically an interface that is meant to be re-implemented by template classes implementing the call and comparison of the function pointers.

It is re-implemented by one of the QSlotObject, QStaticSlotObject or QFunctorSlotObject template class.

Fake Virtual Table

The problem with that is that each instantiation of those object would need to create a virtual table which contains not only pointer to virtual functions but also lot of information we do not need such as RTTI. That would result in lot of superfluous data and relocation in the binaries.

In order to avoid that, QSlotObjectBase was changed not to be a C++ polymorphic class. Virtual functions are emulated by hand.

class QSlotObjectBase {
  QAtomicInt m_ref;
  typedef void (*ImplFn)(int which, QSlotObjectBase* this_,
                         QObject *receiver, void **args, bool *ret);
  const ImplFn m_impl;
protected:
  enum Operation { Destroy, Call, Compare };
public:
  explicit QSlotObjectBase(ImplFn fn) : m_ref(1), m_impl(fn) {}
  inline int ref() Q_DECL_NOTHROW { return m_ref.ref(); }
  inline void destroyIfLastRef() Q_DECL_NOTHROW {
    if (!m_ref.deref()) m_impl(Destroy, this, 0, 0, 0);
  }

  inline bool compare(void **a) { bool ret; m_impl(Compare, this, 0, a, &ret); return ret; }
  inline void call(QObject *r, void **a) {  m_impl(Call,    this, r, a, 0); }
};

The m_impl is a (normal) function pointer which performs the three operations that were previously virtual functions. The "re-implementations" set it to their own implementation in the constructor.

Please do not go in your code and replace all your virtual functions by such a hack because you read here it was good. This is only done in this case because almost every call to connect would generate a new different type (since the QSlotObject has template parameters wich depend on signature of the signal and the slot).

Protected, Public, or Private Signals.

Signals were protected in Qt4 and before. It was a design choice as signals should be emitted by the object when its change its state. They should not be emitted from outside the object and calling a signal on another object is almost always a bad idea.

However, with the new syntax, you need to be able take the address of the signal from the point you make the connection. The compiler would only let you do that if you have access to that signal. Writing &Counter::valueChanged would generate a compiler error if the signal was not public.

In Qt 5 we had to change signals from protected to public. This is unfortunate since this mean anyone can emit the signals. We found no way around it. We tried a trick with the emit keyword. We tried returning a special value. But nothing worked. I believe that the advantages of the new syntax overcome the problem that signals are now public.

Sometimes it is even desirable to have the signal private. This is the case for example in QAbstractItemModel, where otherwise, developers tend to emit signal from the derived class which is not what the API wants. There used to be a pre-processor trick that made signals private but it broke the new connection syntax.
A new hack has been introduced. QPrivateSignal is a dummy (empty) struct declared private in the Q_OBJECT macro. It can be used as the last parameter of the signal. Because it is private, only the object has the right to construct it for calling the signal. MOC will ignore the QPrivateSignal last argument while generating signature information. See qabstractitemmodel.h for an example.

More Template Code

The rest of the code is in qobjectdefs_impl.h and qobject_impl.h. It is mostly standard dull template code.

I will not go into much more details in this article, but I will just go over few items that are worth mentioning.

Meta-Programming List

As pointed out earlier, FunctionPointer::Arguments is a list of the arguments. The code needs to operate on that list: iterate over each element, take only a part of it or select a given item.

That is why there is QtPrivate::List that can represent a list of types. Some helpers to operate on it are QtPrivate::List_Select and QtPrivate::List_Left, which give the N-th element in the list and a sub-list containing the N first elements.

The implementation of List is different for compilers that support variadic templates and compilers that do not.

With variadic templates, it is a template<typename... T> struct List;. The list of arguments is just encapsulated in the template parameters.
For example: the type of a list containing the arguments (int, QString, QObject*) would simply be:

List<int, QString, QObject *>

Without variadic template, it is a LISP-style list: template<typename Head, typename Tail > struct List; where Tail can be either another List or void for the end of the list.
The same example as before would be:

List<int, List<QString, List<QObject *, void> > >

ApplyReturnValue Trick

In the function FunctionPointer::call, the args[0] is meant to receive the return value of the slot. If the signal returns a value, it is a pointer to an object of the return type of the signal, else, it is 0. If the slot returns a value, we need to copy it in arg[0]. If it returns void, we do nothing.

The problem is that it is not syntaxically correct to use the return value of a function that returns void. Should I have duplicated the already huge amount of code duplication: once for the void return type and the other for the non-void? No, thanks to the comma operator.

In C++ you can do something like that:

functionThatReturnsVoid(), somethingElse();

You could have replaced the comma by a semicolon and everything would have been fine.

Where it becomes interesting is when you call it with something that is not void:

functionThatReturnsInt(), somethingElse();

There, the comma will actually call an operator that you even can overload. It is what we do in qobjectdefs_impl.h

template <typename T>
struct ApplyReturnValue {
    void *data;
    ApplyReturnValue(void *data_) : data(data_) {}
};

template<typename T, typename U>
void operator,(const T &value, const ApplyReturnValue<U> &container) {
    if (container.data)
        *reinterpret_cast<U*>(container.data) = value;
}
template<typename T>
void operator,(T, const ApplyReturnValue<void> &) {}

ApplyReturnValue is just a wrapper around a void*. Then it can be used in each helper. This is for example the case of a functor without arguments:

  static void call(Function &f, void *, void **arg) {
      f(), ApplyReturnValue<SignalReturnType>(arg[0]);
  }

This code is inlined, so it will not cost anything at run-time.

Conclusion

This is it for this blog post. There is still a lot to talk about (I have not even mentioned QueuedConnection or thread safety yet), but I hope you found this interresting and that you learned here something that might help you as a programmer.

Qt is well known for its signals and slots mechanism. But how does it work? In this blog post, we will explore the internals of QObject and QMetaObject and discover how signals and slot work under the hood.

In this blog article, I show portions of Qt5 code, sometimes edited for formatting and brevity.

Signals and Slots

First, let us recall how signals and slots look like by showing the official example.

The header looks like this:

class Counter : public QObject
{
    Q_OBJECT
    int m_value;
public:
    int value() const { return m_value; }
public slots:
    void setValue(int value);
signals:
    void valueChanged(int newValue);
};

Somewhere in the .cpp file, we implement setValue()

void Counter::setValue(int value)
{
    if (value != m_value) {
        m_value = value;
        emit valueChanged(value);
    }
}

Then one can use this Counter object like this:

  Counter a, b;
  QObject::connect(&a, SIGNAL(valueChanged(int)),
                   &b, SLOT(setValue(int)));

  a.setValue(12);  // a.value() == 12, b.value() == 12

This is the original syntax that has almost not changed since the beginning of Qt in 1992.

But even if the basic API has not changed since the beginning, its implementation has been changed several times. New features have been added and a lot happened under the hood. There is no magic involved and this blog post will show you how it works.

MOC, the Meta Object Compiler

The Qt signals/slots and property system are based on the ability to introspect the objects at runtime. Introspection means being able to list the methods and properties of an object and have all kinds of information about them such as the type of their arguments.
QtScript and QML would have hardly been possible without that ability.

C++ does not offer introspection support natively, so Qt comes with a tool to provide it. That tool is MOC. It is a code generator (and NOT a preprocessor like some people call it).

It parses the header files and generates an additional C++ file that is compiled with the rest of the program. That generated C++ file contains all the information required for the introspection.

Qt has sometimes been criticized by language purists because of this extra code generator. I will let the Qt documentation respond to this criticism. There is nothing wrong with code generators and the MOC is of a great help.

Magic Macros

Can you spot the keywords that are not pure C++ keywords? signals, slots, Q_OBJECT, emit, SIGNAL, SLOT. Those are known as the Qt extension to C++. They are in fact simple macros, defined in qobjectdefs.h

#define signals public
#define slots /* nothing */

That is right, signals and slots are simple functions: the compiler will handle them them like any other functions. The macros still serve a purpose though: the MOC will see them.

Signals were protected in Qt4 and before. They are becoming public in Qt5 in order to enable the new syntax.

#define Q_OBJECT \
public: \
    static const QMetaObject staticMetaObject; \
    virtual const QMetaObject *metaObject() const; \
    virtual void *qt_metacast(const char *); \
    virtual int qt_metacall(QMetaObject::Call, int, void **); \
    QT_TR_FUNCTIONS /* translations helper */ \
private: \
    Q_DECL_HIDDEN static void qt_static_metacall(QObject *, QMetaObject::Call, int, void **);

Q_OBJECT defines a bunch of functions and a static QMetaObject Those functions are implemented in the file generated by MOC.

#define emit /* nothing */

emit is an empty macro. It is not even parsed by MOC. In other words, emit is just optional and means nothing (except being a hint to the developer).

Q_CORE_EXPORT const char *qFlagLocation(const char *method);
#ifndef QT_NO_DEBUG
# define QLOCATION "\0" __FILE__ ":" QTOSTRING(__LINE__)
# define SLOT(a)     qFlagLocation("1"#a QLOCATION)
# define SIGNAL(a)   qFlagLocation("2"#a QLOCATION)
#else
# define SLOT(a)     "1"#a
# define SIGNAL(a)   "2"#a
#endif

Those macros just use the preprocessor to convert the parameter into a string, and add a code in front.

In debug mode we also annotate the string with the file location for a warning message if the signal connection did not work. This was added in Qt 4.5 in a compatible way. In order to know which strings have the line information, we use qFlagLocation which will register the string address in a table with two entries.

MOC Generated Code

We will now go over portion of the code generated by moc in Qt5.

The QMetaObject

const QMetaObject Counter::staticMetaObject = {
    { &QObject::staticMetaObject, qt_meta_stringdata_Counter.data,
      qt_meta_data_Counter,  qt_static_metacall, 0, 0}
};


const QMetaObject *Counter::metaObject() const
{
    return QObject::d_ptr->metaObject ? QObject::d_ptr->dynamicMetaObject() : &staticMetaObject;
}

We see here the implementation of Counter::metaObject() and Counter::staticMetaObject. They are declared in the Q_OBJECT macro. QObject::d_ptr->metaObject is only used for dynamic meta objects (QML Objects), so in general, the virtual function metaObject() just returns the staticMetaObject of the class.

The staticMetaObject is constructed in the read-only data. QMetaObject as defined in qobjectdefs.h looks like this:

struct QMetaObject
{
    /* ... Skiped all the public functions ... */

    enum Call { InvokeMetaMethod, ReadProperty, WriteProperty, /*...*/ };

    struct { // private data
        const QMetaObject *superdata;
        const QByteArrayData *stringdata;
        const uint *data;
        typedef void (*StaticMetacallFunction)(QObject *, QMetaObject::Call, int, void **);
        StaticMetacallFunction static_metacall;
        const QMetaObject **relatedMetaObjects;
        void *extradata; //reserved for future use
    } d;
};

The d indirection is there to symbolize that all the member should be private. They are not private in order to keep it a POD and allow static initialization.

The QMetaObject is initialized with the meta object of the parent object (QObject::staticMetaObject in this case) as superdata. stringdata and data are initialized with some data explained later in this article. static_metacall is a function pointer initialized to Counter::qt_static_metacall.

Introspection Tables

First, let us analyze the integer data of QMetaObject.

static const uint qt_meta_data_Counter[] = {

 // content:
       7,       // revision
       0,       // classname
       0,    0, // classinfo
       2,   14, // methods
       0,    0, // properties
       0,    0, // enums/sets
       0,    0, // constructors
       0,       // flags
       1,       // signalCount

 // signals: name, argc, parameters, tag, flags
       1,    1,   24,    2, 0x05,

 // slots: name, argc, parameters, tag, flags
       4,    1,   27,    2, 0x0a,

 // signals: parameters
    QMetaType::Void, QMetaType::Int,    3,

 // slots: parameters
    QMetaType::Void, QMetaType::Int,    5,

       0        // eod
};

The first 13 int consists of the header. When there are two columns, the first column is the count and the second column is the index in this array where the description starts.
In this case we have 2 methods, and the methods description starts at index 14.

The method descriptions are composed of 5 int. The first one is the name, it is an index in the string table (we will look into the details later). The second integer is the number of parameters, followed by the index at which one can find the parameter description. We will ignore the tag and flags for now. For each function, moc also saves the return type of each parameter, their type and index to the name.

String Table

struct qt_meta_stringdata_Counter_t {
    QByteArrayData data[6];
    char stringdata[47];
};
#define QT_MOC_LITERAL(idx, ofs, len) \
    Q_STATIC_BYTE_ARRAY_DATA_HEADER_INITIALIZER_WITH_OFFSET(len, \
    offsetof(qt_meta_stringdata_Counter_t, stringdata) + ofs \
        - idx * sizeof(QByteArrayData) \
    )
static const qt_meta_stringdata_Counter_t qt_meta_stringdata_Counter = {
    {
QT_MOC_LITERAL(0, 0, 7),
QT_MOC_LITERAL(1, 8, 12),
QT_MOC_LITERAL(2, 21, 0),
QT_MOC_LITERAL(3, 22, 8),
QT_MOC_LITERAL(4, 31, 8),
QT_MOC_LITERAL(5, 40, 5)
    },
    "Counter\0valueChanged\0\0newValue\0setValue\0"
    "value\0"
};
#undef QT_MOC_LITERAL

This is basically a static array of QByteArray. the QT_MOC_LITERAL macro creates a static QByteArray that references a particular index in the string below.

Signals

The MOC also implements the signals. They are simple functions that just create an array of pointers to the arguments and pass that to QMetaObject::activate. The first element of the array is the return value. In our example it is 0 because the return value is void.
The 3rd parameter passed to activate is the signal index (0 in that case).

// SIGNAL 0
void Counter::valueChanged(int _t1)
{
    void *_a[] = { 0, const_cast<void*>(reinterpret_cast<const void*>(&_t1)) };
    QMetaObject::activate(this, &staticMetaObject, 0, _a);
}

Calling a Slot

It is also possible to call a slot by its index in the qt_static_metacall function:

void Counter::qt_static_metacall(QObject *_o, QMetaObject::Call _c, int _id, void **_a)
{
    if (_c == QMetaObject::InvokeMetaMethod) {
        Counter *_t = static_cast<Counter *>(_o);
        switch (_id) {
        case 0: _t->valueChanged((*reinterpret_cast< int(*)>(_a[1]))); break;
        case 1: _t->setValue((*reinterpret_cast< int(*)>(_a[1]))); break;
        default: ;
        }

The array pointers to the argument is the same format as the one used for the signal. _a[0] is not touched because everything here returns void.

A Note About Indexes.

In each QMetaObject, the slots, signals and other invokable methods of that object are given an index, starting from 0. They are ordered so that the signals come first, then the slots and then the other methods. This index is called internally the relative index. They do not include the indexes of the parents.

But in general, we do not want to know a more global index that is not relative to a particular class, but include all the other methods in the inheritance chain. To that, we just add an offset to that relative index and get the absolute index. It is the index used in the public API, returned by functions like QMetaObject::indexOf{Signal,Slot,Method}

The connection mechanism uses a vector indexed by signals. But all the slots waste space in the vector and there are usually more slots than signals in an object. So from Qt 4.6, a new internal signal index which only includes the signal index is used.

While developing with Qt, you only need to know about the absolute method index. But while browsing the Qt's QObject source code, you must be aware of the difference between those three.

How Connecting Works.

The first thing Qt does when doing a connection is to find out the index of the signal and the slot. Qt will look up in the string tables of the meta object to find the corresponding indexes.

Then a QObjectPrivate::Connection object is created and added in the internal linked lists.

What information needs to be stored for each connection? We need a way to quickly access the connections for a given signal index. Since there can be several slots connected to the same signal, we need for each signal to have a list of the connected slots. Each connection must contain the receiver object, and the index of the slot. We also want the connections to be automatically destroyed when the receiver is destroyed, so each receiver object needs to know who is connected to him so he can clear the connection.

Here is the QObjectPrivate::Connection as defined in qobject_p.h :

struct QObjectPrivate::Connection
{
    QObject *sender;
    QObject *receiver;
    union {
        StaticMetaCallFunction callFunction;
        QtPrivate::QSlotObjectBase *slotObj;
    };
    // The next pointer for the singly-linked ConnectionList
    Connection *nextConnectionList;
    //senders linked list
    Connection *next;
    Connection **prev;
    QAtomicPointer<const int> argumentTypes;
    QAtomicInt ref_;
    ushort method_offset;
    ushort method_relative;
    uint signal_index : 27; // In signal range (see QObjectPrivate::signalIndex())
    ushort connectionType : 3; // 0 == auto, 1 == direct, 2 == queued, 4 == blocking
    ushort isSlotObject : 1;
    ushort ownArgumentTypes : 1;
    Connection() : nextConnectionList(0), ref_(2), ownArgumentTypes(true) {
        //ref_ is 2 for the use in the internal lists, and for the use in QMetaObject::Connection
    }
    ~Connection();
    int method() const { return method_offset + method_relative; }
    void ref() { ref_.ref(); }
    void deref() {
        if (!ref_.deref()) {
            Q_ASSERT(!receiver);
            delete this;
        }
    }
};

Each object has then a connection vector: It is a vector which associates for each of the signals a linked lists of QObjectPrivate::Connection.
Each object also has a reversed lists of connections the object is connected to for automatic deletion. It is a doubly linked list.

Linked lists are used because they allow to quickly add and remove objects. They are implemented by having the pointers to the next/previous nodes within QObjectPrivate::Connection
Note that the prev pointer of the senderList is a pointer to a pointer. That is because we don't really point to the previous node, but rather to the pointer to the next in the previous node. This pointer is only used when the connection is destroyed, and not to iterate backwards. It allows not to have a special case for the first item.

Signal Emission

When we call a signal, we have seen that it calls the MOC generated code which calls QMetaObject::activate.

Here is an annotated version of its implementation from qobject.cpp

void QMetaObject::activate(QObject *sender, const QMetaObject *m, int local_signal_index,
                           void **argv)
{
    activate(sender, QMetaObjectPrivate::signalOffset(m), local_signal_index, argv);
    /* We just forward to the next function here. We pass the signal offset of
     * the meta object rather than the QMetaObject itself
     * It is split into two functions because QML internals will call the later. */
}

void QMetaObject::activate(QObject *sender, int signalOffset, int local_signal_index, void **argv)
{
    int signal_index = signalOffset + local_signal_index;

    /* The first thing we do is quickly check a bit-mask of 64 bits. If it is 0,
     * we are sure there is nothing connected to this signal, and we can return
     * quickly, which means emitting a signal connected to no slot is extremely
     * fast. */
    if (!sender->d_func()->isSignalConnected(signal_index))
        return; // nothing connected to these signals, and no spy

    /* ... Skipped some debugging and QML hooks, and some sanity check ... */

    /* We lock a mutex because all operations in the connectionLists are thread safe */
    QMutexLocker locker(signalSlotLock(sender));

    /* Get the ConnectionList for this signal.  I simplified a bit here. The real code
     * also refcount the list and do sanity checks */
    QObjectConnectionListVector *connectionLists = sender->d_func()->connectionLists;
    const QObjectPrivate::ConnectionList *list =
        &connectionLists->at(signal_index);

    QObjectPrivate::Connection *c = list->first;
    if (!c) continue;
    // We need to check against last here to ensure that signals added
    // during the signal emission are not emitted in this emission.
    QObjectPrivate::Connection *last = list->last;

    /* Now iterates, for each slot */
    do {
        if (!c->receiver)
            continue;

        QObject * const receiver = c->receiver;
        const bool receiverInSameThread = QThread::currentThreadId() == receiver->d_func()->threadData->threadId;

        // determine if this connection should be sent immediately or
        // put into the event queue
        if ((c->connectionType == Qt::AutoConnection && !receiverInSameThread)
            || (c->connectionType == Qt::QueuedConnection)) {
            /* Will basically copy the argument and post an event */
            queued_activate(sender, signal_index, c, argv);
            continue;
        } else if (c->connectionType == Qt::BlockingQueuedConnection) {
            /* ... Skipped ... */
            continue;
        }

        /* Helper struct that sets the sender() (and reset it backs when it
         * goes out of scope */
        QConnectionSenderSwitcher sw;
        if (receiverInSameThread)
            sw.switchSender(receiver, sender, signal_index);

        const QObjectPrivate::StaticMetaCallFunction callFunction = c->callFunction;
        const int method_relative = c->method_relative;
        if (c->isSlotObject) {
            /* ... Skipped....  Qt5-style connection to function pointer */
        } else if (callFunction && c->method_offset <= receiver->metaObject()->methodOffset()) {
            /* If we have a callFunction (a pointer to the qt_static_metacall
             * generated by moc) we will call it. We also need to check the
             * saved metodOffset is still valid (we could be called from the
             * destructor) */
            locker.unlock(); // We must not keep the lock while calling use code
            callFunction(receiver, QMetaObject::InvokeMetaMethod, method_relative, argv);
            locker.relock();
        } else {
            /* Fallback for dynamic objects */
            const int method = method_relative + c->method_offset;
            locker.unlock();
            metacall(receiver, QMetaObject::InvokeMetaMethod, method, argv);
            locker.relock();
        }

        // Check if the object was not deleted by the slot
        if (connectionLists->orphaned) break;
    } while (c != last && (c = c->nextConnectionList) != 0);
}

Conclusion

We saw how connections are made and how signals slots are emitted. What we have not seen is the implementation of the new Qt5 syntax, but that will be for another post.

Update: The part 2 is available.

As a developer, I write code, but I also read a lot of code. Often the code of other people. It is easy to get lost among all the functions, objects or files. What calls what? What does this function exactly do? Where is this variable modified? It is important that as much information as possible can be presented or easily accessible.

Good IDEs are good at displaying that information and ease the navigation in the source code. But often, I want to browse source code which I don't necessarily have on my hard drive. While I look at code on the web, I am very disturbed by the poor browsing experience. This is why I developed an online code browser.

Introducing the Woboq Codebrowser

Inspired by the UI of KDevelop (my favorite IDE) and using clang as a parser, I was able to imitate the KDevelop UI. But there is also a Qt Creator skin, for those who like it sober.

The idea is a bit based on LXR or Mozilla's DXR. But the goal is more to give better visualization and browsing experience rather than a search tool.

Semantic highlighting

Once you have tasted the semantic highlighting, you cannot go back to non properly highlighted code.

In this screen shot one can see the KDevelop color scheme. Green for types, dark yellow for members of the current class. Light color for local variables, with different color depending on the variable, allows you to quickly recognize what is what. Virtual methods are in italic.

When you hover over a symbol, all its use gets highlighted.

Navigation

Click on a symbol brings you to the definition of this symbol. You can browse code like you browse web. Open links in new tabs or go back in your browser history.

Tooltips

That's where it is getting interesting: Most of the symbols have a tooltip reminding you of the type of the symbol. The name of the arguments of the function, and even its default arguments. It will show what it thinks might be the documentation for a given symbol directly in the tooltip.

You can also can see the locations where a given function or variable is used. And if you click on it you get redirected to the right location.

Browse code now!

Woboq is hosting the code of some stuff which we find useful: Qt5 (contains V8, JavascriptCore, ...), Qt4, Qt creator, KDE (most of the modules, plus some extra like Quassel), LLVM/Clang, the linux kernel, glibc.

So if you ever wanted to look deeper in the implementation of some of your program, visit http://code.woboq.org.

Technical Details

The HTML pages are generated using a tool that use the clang library from the LLVM project. It parses the code as if it was compiling it, and visit the AST to annotate every token, and build an index of the uses and definitions.

It currently handles C++ and C.
I did not spend much time on template, or on the preprocessor (macros). So it is still not as good as KDevelop in this respect.

Contact us if you are interested to use it for your own sources.

Update: The source code and licence plan are now available.

SIMD: "Single instruction, multiple data" is a class of instructions present in many CPUs today. For example, on the Intel CPU they are known under the SSE acronym. Those instructions enable more parallelism by operating simultaneously on multiple data.

In this blog post I will present a method for converting UTF-8 text to UTF-16 using SSE4 compiler intrinsics. My goal is also to introduce you to the SIMD intrinsics, if you are not familiar with them yet.

About Intrinsics

CPUs can parallelize computations by using specials instructions that operate simultaneously on multiples values. There are large registers that contain vectors of values. For example, the SSE registers are 128 bits vectors that can contain either 2 double, 4 float, 4 int, 8 short or up to 16 char values. Then, one single instruction can add, substract, multiply, ... all those values and put the result in another vector register.
This is called SIMD for Single Instruction Multiple Data.

Compilers can sometimes automatically vectorize the contents of a simple loop, but they are not really good at it. If the loop is too complex or if it depends on the previous iteration, the compiler will fail to vectorize. (Even if it would be obvious for an human). The vectorization therefore has to be done manually.

This is where compiler intrinsics help us. They are special functions handled directly in the compiler who will emit the instruction we want.

The problem with intrinsics is that they are not really portable. They target a specific architecture. This means you need to write your algorithms for the different CPU architectures you target. Fortunately, the main compilers use the same intrinsics so they are portable across compilers and operating systems.

You are supposed to detect the support of the instruction set at run-time. What is usually done is to write several variants of the function for the different target architecture and have a function pointer that is set to the best variant for the running architecture.

The best documentation I could find is the reference on MSDN. You will have to browse it a bit to see what the available intrinsics are.

The Task: Converting UTF-8 to UTF-16

I got interested in SIMD when Benjamin was using them to speed up QString::fromLatin1, which converts a char* encoded in Latin-1 to UTF-16 (QString's internal encoding). After he explained to me how it works, I was able to speed up some of the drawing code inside Qt.
However in Qt5, we changed the default string encoding from Latin-1 to UTF-8.

UTF-8 can represent all of Unicode while Latin-1 can only represent the alphabet of a few latin based languages. UTF-8 parsing is much more difficult to vectorize because a single code point can be encoded by a variable number of bytes.

This table explains the correspondence between the bits in UTF-8 and UTF-16:

I was wondering if we could improve QString::fromUtf8 using SIMD.
Most of the strings are just ASCII which is easy to vectorize, but I wanted to also try to go ahead and vectorize the more complicated cases as well.
However, the 4 bytes sequences (Supplementary Planes) are very rarely used (most used languages and useful symbols are already in the Basic Multilingual Plane) and also more complicated so I did not vectorize that case for the scope of this blog post. This is left as an exercise for the reader ;-)

Previous work

UTF-8 processing is a very common task. I was expecting that the problem would be already solved. Currently however most libraries and applications still have a scalar implementation. Few use vectorization, but only for ASCII.

There is a library called u8u16. It uses a clever technique by separating the bits in individual vectors. It is processing 128 char at the time in 8 different vectors.
I wanted to try a different approach and work directly on one vector. Working with 16 char at the time allows speeding up the processing of smaller strings.

The Easy Case: ASCII

I'm going to start by explaining the simple case when we only have ASCII. This is very easy to vectorize because for ASCII one byte in UTF-8 is always one byte in UTF-16 with \0 bytes in between.
ASCII is also a very common encoding so it make sense to have a special case for it.

I will start by showing the code and then explaining it

Our function takes a pointer src to the UTF-8 buffer of length len, and a pointer dst to a buffer of size at least len*2 where we will store the UTF-16 output. The function returns the size of the UTF-16 output.

int fromUtf8(const char *src, int len, unsigned short *dst) {
  /* We will process the input 16 bytes at a time,
     so the length must be at least 16. */
    while(len >= 16) {

        /* Load 128 bit into a vector register. We use the 'loadu' intrinsic
           where 'u' stands for unaligned. Loading aligned data is much faster,
           but here we don't know if the source is aligned */
        __m128i chunk = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src));

        /* Detect if it is ASCII by looking if one byte has the high bit set. */
        if (!_mm_movemask_epi8(chunk)) {

            // unpack the first 8 bytes, padding with zeros
            __m128i firstHalf = _mm_unpacklo_epi8(chunk, _mm_set1_epi8(0));
            // and store to the destination
            _mm_storeu_si128(reinterpret_cast<__m128i*>(dst), firstHalf); 

            // do the same with the last 8 bytes
            __m128i secondHalf = _mm_unpackhi_epi8 (chunk, _mm_set1_epi8(0));
            _mm_storeu_si128(reinterpret_cast<__m128i*>(dst+8), secondHalf);

            // Advance
            dst += 16;
            src += 16;
            len -= 16;
            continue;
        }

        // handle the more complicated case when the chunk contains multi-bytes
        // ...
    }
    // handle the few remaining bytes using a classical serial algorith
    // ...
}

The type __m128i represents a vector of 128 bits. It is most likely going to be stored in a SSE register.
Since we don't know the alignment, we load the data with _mm_loadu_si128. There is also the _mm_load_si128 that assumes the memory is aligned on 16 bytes. Because it is faster to operate on aligned data, some algorithms sometimes start with a prologue to handle the first bytes until the rest is aligned. I did not do that here but we could optimize the ASCII processing further by doing so.

Once we have loaded 16 bytes of data into chunk, we need to check if it only contains ASCII. Each ASCII byte should have its most significant bit set to 0. _mm_movemask_epi8 creates a 16bit mask out of most significant bits of our 16 bytes in the vector. If that mask is 0 we only have ASCII in that vector.

The next operation is to expand with zeroes. We will do what is called unpacking. The instructions _mm_unpackhi_epi8 and _mm_unpacklo_epi8 are going to interleave two vectors. Because the result of an unpack is on 32 bytes, there is two operations: one to get the first 16 high bytes, and the other to get the low bytes.
_mm_set1_epi8(0) creates a vector filled with zeroes.

Note that vectors are in little endian, so I show the least significant bytes first

Now, we can just store the result with _mm_storeu_si128. We use the unaligned operation again.

General Algorithm

Now that I have introduced you to the ASCII decoding process, we can move on to the general case, in which we might have multiple bytes for one character.

The algorithm proceeds with the following steps:

1. First, we will classify each bytes so we know where the values in the sequence are and how long they are. This information will be stored in vectors that we can use to mask so operations we do later operate only on a certain category of bytes.
2. Then, we will operate on the data to to find the actual Unicode value behind each character. They will be stored at the location of the last byte of a sequence, into two different vectors: one for the low bits and one for the high bits.
3. Then we need to shuffle the vectors to remove the possible "gaps" left by multi-bytes sequences.
4. Finally, we can store the result and advance the pointers.

I will explain every step in more detail.

In our example, I will decode the string x≤(α+β)²γ², which is represented in UTF-8 as x\e2\89\a4(\ce\b1+\ce\b2)\c2\b2\ce\b3\c2\b2. The "≤" is represented by 3 bytes, the Greek letters by two bytes, and the "²" also by two bytes. The total length is 17 bytes, so the last byte will be truncated when loaded into the vector

Classification

At the end of this step, we will have the following vectors:

• mask tells which bit needs to be set to 0 in order to only keep the bits that are going to end up in the Unicode value. Bits of mask that are set will be unset from the data.
• count is zero if the corresponding byte is ASCII or a continuation byte. If the corresponding byte is starting a sequence, then it is the number of bytes in that sequence.
• counts represents (for each byte) how many bytes are remaining in this sequence. It is 0 for ASCII, 1 for the last byte in a sequence, and so on.

We start by finding count and mask. I do that by computing a vector that I called state, in which the 5 higher bits are the mask and the 3 lower bits represent the count.

We will use the _mm_cmplt_epi8 instruction. It compares two vectors of 16 signed char, and returns a mask vector where each byte is equal to 0xff or 0x00 depending of the result of the comparison between the corresponding chars.
We can then use this mask as input to the _mm_blendv_epi8 SSE4 instruction. It takes 3 input arguments: two vectors and a mask. It returns a vector that has the bytes of the second input where the most significant bit of the corresponding byte of the mask is one and the bytes of the first input otherwise. In other words, for each 16 bytes of the vectors, we have:
output[i] = mask[i] & 0x80 ? input2[i] : input1[i]

The problem of _mm_cmplt_epi8 is that it only works on signed bytes. That is why we add 0x80 to everything simulate unsigned comparison.

    // The state for ascii or contiuation bit: mask the most significant bit.
    __m128i state = _mm_set1_epi8(0x0 | 0x80);

    // Add an offset of 0x80 to work around the fact that we don't have
    // unsigned comparison
    __m128i chunk_signed = _mm_add_epi8(chunk, _mm_set1_epi8(0x80));

    // If the bytes are greater or equal than 0xc0, we have the start of a
    // multi-bytes sequence of at least 2.
    // We use 0xc2 for error detection, see later.
    __m128i cond2 = _mm_cmplt_epi8( _mm_set1_epi8(0xc2-1 -0x80), chunk_signed);

    state = _mm_blendv_epi8(state , _mm_set1_epi8(0x2 | 0xc0),  cond2);

    __m128i cond3 = _mm_cmplt_epi8( _mm_set1_epi8(0xe0-1 -0x80), chunk_signed);

    // We could optimize the case when there is no sequence logner than 2 bytes.
    // But i did not do it in this version.
    //if (!_mm_movemask_epi8(cond3)) { /* process max 2 bytes sequences */ }

    state = _mm_blendv_epi8(state , _mm_set1_epi8(0x3 | 0xe0),  cond3);
    __m128i mask3 = _mm_slli_si128(cond3, 1);

    // In this version I do not handle sequences of 4 bytes. So if there is one
    // we break and do a classic processing byte per byte.
    __m128i cond4 = _mm_cmplt_epi8(_mm_set1_epi8(0xf0-1 -0x80), chunk_signed);
    if (_mm_movemask_epi8(cond4)) { break; }

    // separate the count and mask from the state vector
    __m128i count =  _mm_and_si128(state, _mm_set1_epi8(0x7));
    __m128i mask = _mm_and_si128(state, _mm_set1_epi8(0xf8));

From count I will be able to compute counts. I will use _mm_subs_epu8 which substracts two vector of unsigned bytes and saturate (so if you underflow, it sets 0). Then I shift that result, and add it to the count. Then we do the same, but we substract 2 and shift the result two bytes.

_mm_slli_si128 shifts a vector on the left by the given number of bytes. Notice again that we work on little endian. So that is why I shifted right on the picture, but left in the code.

    //substract 1, shift 1 byte and add
    __m128i count_subs1 = _mm_subs_epu8(count, _mm_set1_epi8(0x1));
    __m128i counts = _mm_add_epi8(count, _mm_slli_si128(count_subs1, 1));

    //substract 2, shift 2 bytes and add
    counts = _mm_add_epi8(counts, _mm_slli_si128(
                    _mm_subs_epu8(counts, _mm_set1_epi8(0x2)), 2));

Processing

The goal of this step is to end up with the final unicode bytes values. They will be split into two vectors (one for the high bytes and one for the low bytes). We keep them in the location of the last byte of a sequence, which means that there will be "gaps" in the vector.

There is no instructions that shift the whole vector by an arbitrary number of bits. It is only possible to shift by an integer number of bytes with _mm_slli_si128 or _mm_srli_si128. The instructions that shift by an abitrary number of bits only work on vector with elements of 16 or 32 bits. No 8 bits elements. We will use what we have and will first shift by one byte (in chunk_right) and then shift and mask to get what we want.

    // Mask away our control bits with ~mask (and not)
    chunk = _mm_andnot_si128(mask , chunk);
    // from now on, we only have usefull bits

    // shift by one byte on the left
    __m128i chunk_right = _mm_slli_si128(chunk, 1);

    // If counts == 1,  compute the low byte using 2 bits from chunk_right
    __m128i chunk_low = _mm_blendv_epi8(chunk,
                _mm_or_si128(chunk, _mm_and_si128(
                    _mm_slli_epi16(chunk_right, 6), _mm_set1_epi8(0xc0))),
                _mm_cmpeq_epi8(counts, _mm_set1_epi8(1)) );

    // in chunk_high, only keep the bits if counts == 2
    __m128i chunk_high = _mm_and_si128(chunk ,
                                       _mm_cmpeq_epi8(counts, _mm_set1_epi8(2)));
    // and shift that by 2 bits on the right.
    // (no unwanted bits are comming because of the previous mask)
    chunk_high = _mm_srli_epi32(chunk_high, 2);

    // Add the bits from the bytes for which counts == 3
    __m128i mask3 = _mm_slli_si128(cond3, 1); //re-use cond3 (shifted)
    chunk_high = _mm_or_si128(chunk_high, _mm_and_si128(
        _mm_and_si128(_mm_slli_epi32(chunk_right, 4), _mm_set1_epi8(0xf0)),
                              mask3));

Shuffle

Now, we need to get rid of the gaps in our strings. We will start by computing, for each byte, the number of bytes we need to shift. We notice that count_subs1is the number of bytes we need to skip for a given sequence. We will then accumulate all those number in order to get the number of bytes we need to shift each bytes. Then we reset to zero all the bytes that are supposed to go away.

    __m128i shifts = count_subs1;
    shifts = _mm_add_epi8(shifts, _mm_slli_si128(shifts, 2));
    shifts = _mm_add_epi8(shifts, _mm_slli_si128(shifts, 4));
    shifts = _mm_add_epi8(shifts, _mm_slli_si128(shifts, 8));

    // keep only if the corresponding byte should stay
    // that is, if counts is 1 or 0  (so < 2)
    shifts  = _mm_and_si128 (shifts , _mm_cmplt_epi8(counts, _mm_set1_epi8(2)));

We will shift the shift vector so the meaning of it becomes the number of bytes that are separating a byte in its final location from the byte in its original location.

The maximum we can shift is 10 (if everything is a 3 bytes sequence). We can then compute in 4 steps.
We will shift the interesting bit of the vector shift, so it can be used to control the blend operation.

        shifts = _mm_blendv_epi8(shifts, _mm_srli_si128(shifts, 1),
                                 _mm_srli_si128(_mm_slli_epi16(shifts, 7) , 1));

        shifts = _mm_blendv_epi8(shifts, _mm_srli_si128(shifts, 2),
                                 _mm_srli_si128(_mm_slli_epi16(shifts, 6) , 2));

        shifts = _mm_blendv_epi8(shifts, _mm_srli_si128(shifts, 4),
                                _mm_srli_si128(_mm_slli_epi16(shifts, 5) , 4));

        shifts = _mm_blendv_epi8(shifts, _mm_srli_si128(shifts, 8),
                                 _mm_srli_si128(_mm_slli_epi16(shifts, 4) , 8));

Then we can then know, for each byte in the final vector, its location in the original vector. With that information, we can use _mm_shuffle_epi8 (SSSE3) to shuffle the vector and remove the "gaps". Then we can unpack the result and store it, and we are almost done.

    __m128i shuf = _mm_add_epi8(shifts,
            _mm_set_epi8(15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0));

    // Remove the gaps by shuffling
    shuffled_low = _mm_shuffle_epi8(chunk_low, shuf);
    shuffled_high = _mm_shuffle_epi8(chunk_high, shuf);

    // Now we can unpack and store
    __m128i utf16_low = _mm_unpacklo_epi8(shuffled_low, shuffled_high);
    __m128i utf16_high = _mm_unpackhi_epi8(shuffled_low, shuffled_high);
    _mm_storeu_si128(reinterpret_cast<__m128i*>(dst), utf16_low);
    _mm_storeu_si128(reinterpret_cast<__m128i*>(dst+8) , utf16_high);

Error Detection

So far we have only considered valid UTF-8. But a conform UTF-8 decoder should also handle broken UTF-8. It should detect the following error cases:

1. Not enough, misplaced or too many continuation characters.
2. Overlong forms: If a sequence decodes to a value that could have been encoded in a shorter sequence. For example the quote must be encoded in its ASCII form (\22) and not in a multi-byte sequence (\c0\a2 or \e0\80\a2). Not forbidding those forms could lead to security problems, e.g. if the the quote where escaped in the utf-8 input string, but appears again later when decoded.
3. Invalid or reserved Unicode code point (for example, the one reserved for UTF-16 surrogates)

Since error is not the common case, what I do in that case is to break, and let the rest be handled by the sequential algorithm (Just like when we have 4 bytes sequences).

    // ASCII characters (and only them) should have the
    // corresponding byte of counts equal 0.
    if (asciiMask ^ _mm_movemask_epi8(_mm_cmpgt_epi8(counts, _mm_set1_epi8(0))))
            break;

    // The difference between a byte in counts and the next one should be negative,
    // zero, or one. Any other value means there is not enough continuation bytes.
    if (_mm_movemask_epi8(_mm_cmpgt_epi8(_mm_sub_epi8(_mm_slli_si128(counts, 1),
                counts), _mm_set1_epi8(1))))
            break;

    // For the 3 bytes sequences we check the high byte to prevent
    // the over long sequence (0x00-0x07) or the UTF-16 surrogate (0xd8-0xdf)
    __m128i high_bits = _mm_and_si128(chunk_high, _mm_set1_epi8(0xf8));
    if (!_mm_testz_si128(mask3,
                _mm_or_si128(_mm_cmpeq_epi8(high_bits,_mm_set1_epi8(0x00)) ,
                            _mm_cmpeq_epi8(high_bits,_mm_set1_epi8(0xd8))) ))
        break;

    // Check for a few more invalid unicode using range comparison and _mm_cmpestrc
    const int check_mode = _SIDD_UWORD_OPS | _SIDD_CMP_RANGES;
    if (_mm_cmpestrc( _mm_cvtsi64_si128(0xfdeffdd0fffffffe), 4,
                     utf16_high, 8, check_mode) |
        _mm_cmpestrc( _mm_cvtsi64_si128(0xfdeffdd0fffffffe), 4,
                      utf16_low, 8, check_mode))
            break;

Advance Pointers

Now we are almost done. We just need to advance the source and destination pointers. The source pointer usually advance by 16, unless it ends in the middle of a sequence. In that case, we need to roll back 1 or 2 chars. For that, we look at the end of the counts vector.
Then we can see how much we need to advance the destination by looking at the end of the shift vector.

    int s = _mm_extract_epi32(shifts, 3);
    int dst_advance = source_advance - (0xff & (s >> 8*(3 - 16 + source_advance)));

    int c = _mm_extract_epi16(counts, 7);
    int source_advance = !(c & 0x0200) ? 16 : !(c & 0x02) ? 15 : 14;

    dst += dst_advance;
    src += source_advance;

Putting it all together

Now we are done, you can see the result there

I mixed all the operations to limit the dependencies between the operations that are next to each other to achieve better pipeline performance. I am surprised that the compiler did not do that for me.

Benchmarks Results

We beat u8u16 on small strings but it is better on large strings because it processes more bytes at the same time. We are still better than any scalar processing.

I added some debug output in the implementation of QString::fromUtf8 in Qt 4.8 and ran few KDE applications translated in French. The average size of strings that goes through this function is 9 characters. More than 50% of the string are less than 2 character long, 85% are less than 16, and 13% are between 16 and 128 characters long. 98% of the strings where only composed of ASCII. The rest was mainly having 2 bytes sequences and very few 3 bytes sequences. There is no sequence of 4 bytes.
Also consider that the processing of UTF-8 is probably far from being the bottleneck of any application that does something useful.

So was my exercise useless? Maybe in practice, but I think there is still something to learn from this that can be applied to another algorithm that might be a bottleneck.

In an attempt to bring the things we like to you, we are offering accounts on our Woboq Quasselcore.

In case you have not heard of Quassel before: Quassel is a distributed IRC client where the Quassel core stays connected to your IRC channels and you can use any of your computers to connect to the core and have your chat history accessible.

To find our more, check out our Woboq Quassel Hosting page or visit us on #woboquassel on irc.freenode.net.

As a small side project, we are also trying to bring the Quassel IRC experience to the iPad. More about this will follow here :-) Update: Quassel for iPad has arrived!

C++11 is the name of the current version of the C++ standard, which brings many new features to the language.

Qt 4.8 was the first version of Qt that started to make use of some of the new C++11 features in its API. I wrote a blog post about C++11 in Qt 4.8 before the 4.8 release which I won't repeat here.

In Qt5, we make use of even more features. I will go into detail about some of them in this post.

Lambda expressions for slots

Lambda expressions are a new syntax in C++11 which allow to declare anonymous functions. Anonymous functions can be used in order to use small functions as parameter without requiring to explicitly declare them.
The previous way to write functors using operator() in a struct was requiring a lot of boilerplate code.

It was already possible in Qt 4.8 to use lambda expressions in some QtConcurrent functions. Now it is even possible to use them as slots using the new connect syntax.

Remember the time when you had to write a one-line function for your slot. Now you can write it right in place. It is much more readable to have the function directly where you reference it:

 connect(sender, &Sender::valueChanged, [=](const QString &newValue) {
        receiver->updateValue("senderValue", newValue);
    });

Lambda functions are supported since MSVC 2010, GCC 4.5, and clang 3.1.

Unicode literal

In C++11, you can generate UTF-16 by writing u"MyString". This is used by Qt to implement QStringLiteral which is a macro that initializes the QString at compile time without run-time overhead.

    QString someString = QStringLiteral("Hello");

See my previous blog about QStringLiteral.

Constant expressions: constexpr

This new constexpr C++11 keyword can be added to annotate some inline functions to specify that they could be computed at compile time. In Qt 5, we introduced Q_DECL_CONSTEXPR which is defined to constexpr when the compiler supports it, or nothing otherwise.

We also annotated some of the Qt functions that made sense (for example QFlags), which allows them to be used in constant expressions.

enum SomeEnum { Value1, Value2, Value3 };
Q_DECLARE_OPERATORS_FOR_FLAGS(QFlags<SomeEnum>)
// The previous line declares
// Q_DECL_CONSTEXPR QFlags<SomeValue> operator|(SomeValue,SomeValue) {...}

int someFunction(QFlags<SomeEnum> value) {
    switch (value) {
        case SomeEnum::Value1:
            return 1;
        case SomeEnum::Value2:
            return 2;
        case SomeEnum::Value1 | SomeEnum::Value3:
        // Only possible with C++11 and because QFlags operators are constexpr
        // Previously this line would call
        //        QFlags<SomeValue> operator|(SomeValue,SomeValue)
        // that would have thrown an error because only compiler constants
        // are allowed as case satement

            return 3;
        default:
            return 0;
    }
}

(Notice also here that I used SomeEnum:: in front of the values, which is allowed in C++11, but was not allowed before)

static_assert

C++11 helps producing better error messages when something wrong can be detected at compile time using static_assert. In Qt5 we introduced the macros Q_STATIC_ASSERT, and Q_STATIC_ASSERT_X That will use static_assert if available, or some other template trick if not.

Qt uses that macro already quite a lot in order to provide better compiler errors when the API is not used properly.

Override and final

Have you ever had code that did not work because you had a typo in the name of a virtual function you re-implemented? (Or forgot that damn const at the end)

You can now annotate the functions that are meant to override virtual functions with Q_DECL_OVERRIDE.

That macro will be substituted to the new "override" attribute if the compilers supports it, or to nothing otherwise. If you compile your code with C++11 enabled compiler you will get an error if you have a typo, or if you changed a virtual method while re-factoring.

class MyModel : public QStringListModel {
    //...
protected:
     Qt::ItemFlags flags (const QModelIndex & index)  Q_DECL_OVERRIDE;
};

And because we forgot the const that will produce errors such as:

mymodel.h:15: error: `Qt::ItemFlags MyModel::flags(const QModelIndex&)`
 marked override, but does not override

There is also Q_DECL_FINAL that is substituted to the final attribute which specifies a virtual function cannot be overridden.

Deleted member

The new macro Q_DECL_DELETE expands to = delete for compilers that support deleted functions. This us useful to give better compiler errors to avoid common mistakes.

Deleted functions are used to explicitly delete the function that would be otherwise ceated automatically by the compiler (such as a default constructor, or a copy assignement operator). Deleted functions may not be called and a compiler error is thrown if they are used.

We use it for example in the Q_DISABLE_COPY macro. Before, the trick was to keep those members private, but the error message was not as good.

Rvalue References and Move Constructors

In the Qt 4.8 article I already explained what rvalue references are about.

Due to a change of the internals of the reference counting of our shared class in Qt5, it was possible to add a move constructor for many of them.

Conclusion

MSVC does not require any special flags and enables the C++11 features by default, but GCC or Clang require -std=c++0x.

By default, Qt5 itself will be compiled with the C++11 flags on compilers that need it.

If you use qmake, you can add that line to your .pro file (Qt5):

CONFIG += c++11

(In Qt4, it should be something like: gcc:CXXFLAGS += -std=c++0x)

And now you can enjoy all the nice features of C++11. (It is already worth doing it only for being able to use auto)

QStringLiteral is a new macro introduced in Qt 5 to create QString from string literals. (String literals are strings inside "" included in the source code). In this blog post, I explain its inner working and implementation.

Summary

Let me start by giving a guideline on when to use it: If you want to initialize a QString from a string literal in Qt5, you should use:

• Most of the cases: QStringLiteral("foo") if it will actually be converted to QString
• QLatin1String("foo") if it is use with a function that has an overload for QLatin1String. (such as operator==, operator+, startWith, replace, ...)

I have put this summary at the beginning for the ones that don't want to read the technical details that follow.

Read on to understand how QStringLiteral works

Reminder on how QString works

QString, as many classes in Qt, is an implicitly shared class. Its only member is a pointer to the 'private' data. The QStringData is allocated with malloc, and enough room is allocated after it to put the actual string data in the same memory block.

// Simplified for the purpose of this blog
struct QStringData {
  QtPrivate::RefCount ref; // wrapper around a QAtomicInt
  int size; // size of the string
  uint alloc : 31; // amount of memory reserved after this string data
  uint capacityReserved : 1; // internal detail used for reserve()

  qptrdiff offset; // offset to the data (usually sizeof(QSringData))

  inline ushort *data()
  { return reinterpret_cast<ushort *>(reinterpret_cast<char *>(this) + offset); }
};

// ...

class QString {
  QStringData *d;
public:
  // ... public API ...
};

The offset is a pointer to the data relative to the QStringData. In Qt4, it used to be an actual pointer. We'll see why it has been changed.

The actual data in the string is stored in UTF-16, which uses 2 bytes per character.

Literals and Conversion

Strings literals are the strings that appears directly in the source code, between quotes.
Here are some examples. (suppose action, string, and filename are QString

    o->setObjectName("MyObject");
    if (action == "rename")
        string.replace("%FileName%", filename);

In the first line, we call the function QObject::setObjectName(const QString&). There is an implicit conversion from const char* to QString, via its constructor. A new QStringData is allocated with enough room to hold "MyObject", and then the string is copied and converted from UTF-8 to UTF-16.

The same happens in the last line where the function QString::replace(const QString &, const QString &) is called. A new QStringData is allocated for "%FileName%".

Is there a way to prevent the allocation of QStringData and copy of the string?

Yes, one solution to avoid the costly creation of a temporary QString object is to have overload for common function that takes const char* parameter.
So we have those overloads for operator==

bool operator==(const QString &, const QString &);
bool operator==(const QString &, const char *);
bool operator==(const char *, const QString &) 

The overloads do not need to create a new QString object for our literal and can operate directly on the raw char*.

Encoding and QLatin1String

In Qt5, we changed the default decoding for the char* strings to UTF-8. But many algorithms are much slower with UTF-8 than with plain ASCII or latin1

Hence you can use QLatin1String, which is just a thin wrapper around char * that specify the encoding. There are overloads taking QLatin1String for functions that can opperate or the raw latin1 data directly without conversion.

So our first example now looks like:

    o->setObjectName(QLatin1String("MyObject"));
    if (action == QLatin1String("rename"))
        string.replace(QLatin1String("%FileName%"), filename);

The good news is that QString::replace and operator== have overloads for QLatin1String. So that is much faster now.

In the call to setObjectName, we avoided the conversion from UTF-8, but we still have an (implicit) conversion from QLatin1String to QString which has to allocate the QStringData on the heap.

Introducing QStringLiteral

Is it possible to avoid the allocation and copy of the string literal even for the cases like setObjectName? Yes, that is what QStringLiteral is doing.

This macro will try to generate the QStringData at compile time with all the field initialized. It will even be located in the .rodata section, so it can be shared between processes.

We need two languages feature to do that:

1. The possibility to generate UTF-16 at compile time:
   On Windows we can use the wide char L"String". On Unix we are using the new C++11 Unicode literal: u"String". (Supported by GCC 4.4 and clang.)
2. The ability to create static data from expressions.
   We want to be able to put QStringLiteral everywhere in the code. One way to do that is to put a static QStringData inside a C++11 lambda expression. (Supported by MSVC 2010 and GCC 4.5) (And we also make use of the GCC __extension__ ({ }) Update: The support for the GCC extension was removed before the beta because it does not work in every context lambas are working, such as in default functions arguments)

Implementation

We will need need a POD structure that contains both the QStringData and the actual string. Its structure will depend on the method we use to generate UTF-16.

The code bellow was extracted from qstring.h, with added comments and edited for readability.

/* We define QT_UNICODE_LITERAL_II and declare the qunicodechar
   depending on the compiler */
#if defined(Q_COMPILER_UNICODE_STRINGS)
   // C++11 unicode string
   #define QT_UNICODE_LITERAL_II(str) u"" str
   typedef char16_t qunicodechar;
#elif __SIZEOF_WCHAR_T__ == 2
   // wchar_t is 2 bytes  (condition a bit simplified)
   #define QT_UNICODE_LITERAL_II(str) L##str
   typedef wchar_t qunicodechar;
#else
   typedef ushort qunicodechar; // fallback
#endif

// The structure that will contain the string.
// N is the string size
template <int N>
struct QStaticStringData
{
    QStringData str;
    qunicodechar data[N + 1];
};

// Helper class wrapping a pointer that we can pass to the QString constructor
struct QStringDataPtr
{ QStringData *ptr; };

#if defined(QT_UNICODE_LITERAL_II)
// QT_UNICODE_LITERAL needed because of macro expension rules
# define QT_UNICODE_LITERAL(str) QT_UNICODE_LITERAL_II(str)
# if defined(Q_COMPILER_LAMBDA)

#  define QStringLiteral(str) \
    ([]() -> QString { \
        enum { Size = sizeof(QT_UNICODE_LITERAL(str))/2 - 1 }; \
        static const QStaticStringData<Size> qstring_literal = { \
            Q_STATIC_STRING_DATA_HEADER_INITIALIZER(Size), \
            QT_UNICODE_LITERAL(str) }; \
        QStringDataPtr holder = { &qstring_literal.str }; \
        const QString s(holder); \
        return s; \
    }()) \

# elif defined(Q_CC_GNU)
// Use GCC To  __extension__ ({ }) trick instead of lambda
// ... <skiped> ...
# endif
#endif

#ifndef QStringLiteral
// no lambdas, not GCC, or GCC in C++98 mode with 4-byte wchar_t
// fallback, return a temporary QString
// source code is assumed to be encoded in UTF-8
# define QStringLiteral(str) QString::fromUtf8(str, sizeof(str) - 1)
#endif

Let us simplify a bit this macro and look how the macro would expand

o->setObjectName(QStringLiteral("MyObject"));
// would expand to: 
o->setObjectName(([]() {
        // We are in a lambda expression that returns a QStaticString

        // Compute the size using sizeof, (minus the null terminator)
        enum { Size = sizeof(u"MyObject")/2 - 1 };

        // Initialize. (This is static data initialized at compile time.)
        static const QStaticStringData<Size> qstring_literal =
        { { /* ref = */ -1, 
            /* size = */ Size, 
            /* alloc = */ 0, 
            /* capacityReserved = */ 0, 
            /* offset = */ sizeof(QStringData) },
          u"MyObject" };

         QStringDataPtr holder = { &qstring_literal.str };
         QString s(holder); // call the QString(QStringDataPtr&) constructor
         return s;
    }()) // Call the lambda
  );

The reference count is initialized to -1. A negative value is never incremented or decremented because we are in read only data.

One can see why it is so important to have an offset (qptrdiff) rather than a pointer to the string (ushort*) as it was in Qt4. It is indeed impossible to put pointer in the read only section because pointers might need to be relocated at load time. That means that each time an application or library, the OS needs to re-write all the pointers addresses using the relocation table.

Results

For fun, we can look at the assembly generated for a very simple call to QStringLiteral. We can see that there is almost no code, and how the data is laid out in the .rodata section

We notice the overhead in the binary. The string takes twice as much memory since it is encoded in UTF-16, and there is also a header of sizeof(QStringData) = 24. This memory overhead is the reason why it still makes sense to still use QLatin1String when the function you are calling has an overload for it.

QString returnAString() {
    return QStringLiteral("Hello");
}

Compiled with g++ -O2 -S -std=c++0x (GCC 4.7) on x86_64

    .text
    .globl  _Z13returnAStringv
    .type   _Z13returnAStringv, @function
_Z13returnAStringv:
    ; load the address of the QStringData into %rdx
    leaq    _ZZZ13returnAStringvENKUlvE_clEvE15qstring_literal(%rip), %rdx
    movq    %rdi, %rax
    ; copy the QStringData from %rdx to the QString return object
    ; allocated by the caller.  (the QString constructor has been inlined)
    movq    %rdx, (%rdi)
    ret
    .size   _Z13returnAStringv, .-_Z13returnAStringv
    .section    .rodata
    .align 32
    .type   _ZZZ13returnAStringvENKUlvE_clEvE15qstring_literal, @object
    .size   _ZZZ13returnAStringvENKUlvE_clEvE15qstring_literal, 40
_ZZZ13returnAStringvENKUlvE_clEvE15qstring_literal:
    .long   -1   ; ref
    .long   5    ; size
    .long   0    ; alloc + capacityReserved 
    .zero   4    ; padding
    .quad   24   ; offset
    .string "H"  ; the data. Each .string add a terminal '\0'
    .string "e"
    .string "l"
    .string "l"
    .string "o"
    .string ""
    .string ""
    .zero   4

Conclusion

I hope that now that you have read this you will have a better understanding on where to use and not to use QStringLiteral.
There is another macro QByteArrayLiteral, which work exactly on the same principle but creates a QByteArray.

Update: See also the internals of QMutex and more C++11 features in Qt5.

Qt5 alpha has been released. One of the features which I have been working on is a new syntax for signals and slot. This blog entry will present it.

Previous syntax

Here is how you would connect a signal to a slot:

connect(sender, SIGNAL(valueChanged(QString,QString)),
        receiver, SLOT(updateValue(QString)) );

What really happens behind the scenes is that the SIGNAL and SLOT macros will convert their argument to a string. Then QObject::connect() will compare those strings with the introspection data collected by the moc tool.

What's the problem with this syntax?

While working fine in general, we can identify some issues:

• No compile time check: All the checks are done at run-time by parsing the strings. That means if you do a typo in the name of the signal or the slot, it will compile but the connection will not be made, and you will only notice a warning in the standard output.
• Since it operates on the strings, the type names of the slot must match exactly the ones of the signal. And they also need to be the same in the header and in the connect statement. This means it won't work nicely if you want to use typedef or namespaces

New syntax: using function pointers

In the upcoming Qt5, an alternative syntax exist. The former syntax will still work. But you can now also use this new way of connecting your signals to your slots:

connect(sender, &Sender::valueChanged,
        receiver, &Receiver::updateValue );

Which one is the more beautiful is a matter of taste. One can quickly get used to the new syntax.

So apart from the aesthetic point of view, let us go over some of the things that it brings us:

Compile-time checking

You will get a compiler error if you misspelled the signal or slot name, or if the arguments of your slot do not match those from the signal.
This might save you some time while you are doing some re-factoring and change the name or arguments of signals or slots.

An effort has been made, using static_assert to get nice compile errors if the arguments do not match or of you miss a Q_OBJECT

Arguments automatic type conversion

Not only you can now use typedef or namespaces properly, but you can also connect signals to slots that take arguments of different types if an implicit conversion is possible

In the following example, we connect a signal that has a QString as a parameter to a slot that takes a QVariant. It works because QVariant has an implicit constructor that takes a QString

class Test : public QObject 
{ Q_OBJECT
public:
    Test() {
        connect(this, &Test::someSignal, this, &Test::someSlot);
    }
signals:
    void someSignal(const QString &);
public:
    void someSlot(const QVariant &);
};

Connecting to any function

As you might have seen in the previous example, the slot was just declared as public and not as slot. Qt will indeed call directly the function pointer of the slot, and will not need moc introspection anymore. (It still needs it for the signal)

But what we can also do is connecting to any function or functor:

static void someFunction() {
    qDebug() << "pressed";
}
// ... somewhere else
    QObject::connect(button, &QPushButton::clicked, someFunction);

This can become very powerful when you associate that with boost or tr1::bind.

C++11 lambda expressions

Everything documented here works with the plain old C++98. But if you use compiler that supports C++11, I really recommend you to use some of the language's new features. Lambda expressions are supported by at least MSVC 2010, GCC 4.5, clang 3.1. For the last two, you need to pass -std=c++0x as a flag.

You can then write code like:

void MyWindow::saveDocumentAs() {
    QFileDialog *dlg = new QFileDialog();
    dlg->open();
    QObject::connect(dlg, &QDialog::finished, [=](int result) {
        if (result) {
            QFile file(dlg->selectedFiles().first());
            // ... save document here ...
        }
        dlg->deleteLater();
    });
}

This allows you to write asynchronous code very easily.

Update: Also have a look what other C++11 features Qt5 offers.

So what now?

It is time to try it out. Check out the alpha and start playing. Don't hesistate to report bugs.

While I was working on Qt at Nokia I spent a lot of time reviewing patches from colleagues.

Peer reviewing is an important step, and there is a reason why we do it in Qt.
Code in libraries (such as Qt) requires much greater care than application code.
The main reason is that you need to maintain compatibility for your users. You want to avoid regressions in your library, so code that works continues to work, even if the user did some hack or trick. For the user of the library, it has great value to be able to upgrade it without too much effort. A subtle behaviour difference in the library might introduce a bug in the application that might be really hard to track in the millions of old lines of code of the application.

Reviewing changes starts by looking for obvious mistakes and incompatibilities with the project guidelines:

• Is the code style correct?
• Is the code (auto)tested?
• Are the new APIs properly documented?
• Does the code follow the project policy? (Thread safety, no static global objects, no exported symbols without prefix, No use of features not supported by a supported compiler, ...)

The hard part of the review is seeing the things that the author of the patch could not possibly know.
It takes someone already familiar with the code to do the review. Someone who knows how the code interacts with the other parts, someone who knows the rationale behind some of the part of the code.
Even if you are familiar with the code, you almost always need to open the IDE and browse the existing code and the code it interacts too. Making a review by just looking at the patch may be possible for trivial compilation fix, but in almost all case of non trivial code, you will need to open the relevant files on your IDE.

There is so many things to check and to be aware of: code calling virtual functions, code used for other use cases than the one of the bug report, ...
You also need to think hard about what the patch might break and what the behaviour changes are.

As an example, let us take someone who does a change in a QComboBox to fix a usability glitch on Windows. Someone not familiar with QComboBox might not know that it behaves very differently depending on the style. The reviewer will likely to be able to spot that a change will "break" that style.

Another example is someone who identifies that a bug is caused by one wrong condition in an if clause. Changing seems to fix the problem. But why was that condition there? If the code is not commented and the history of the code shows it is very old, no relevant information can be deduced. The reviewer should be the one who knows that this condition is there to test a very specific use case that might or might not be required anymore.

Inside Nokia we used to have an internal Pastebin website for putting up patches. Nowadays the Qt Project is using a much more advanced process for reviewing.

You may want to read this blog if you want to understand the internals of QMutex or if you are interested in lock-free algorithms and want to discover one. You do not need to read or understand this article if you just want to use QMutex in your application. I found implementing QMutex interesting and therefore I want to share the knowledge.

I am going to show and explain a simplified version of the real code. If you want to see the real code, you can just browse the Qt source code. This article will help you to understand the concepts.

In order to better understand this article, you need to know some basics of lock-free programming. If you understand what the ABA problem is and know how to make a lock-free stack, you are probably fine. Otherwise I recommend reading my previous post as an introduction.

The contents of this blog entry are taken from the last part of my presentation at the Qt Developer days 2011.

QMutex

Before understanding how something works, one needs to understand what it does first. Here is the basic interface we are going to analyze.

class QMutex {
public:
    void lock();
    void unlock();
    bool tryLock();
private:
    //...
};

What it does is unsurprising. If you don't guess what those functions do, you can read it in the QMutex documentation.

Motivation

We want QMutex to be as efficient as possible. QMutex in Qt 4.8 is already quite fast because it is already using atomic operations. So don't expect differences in speed between Qt 4.8 and Qt 5. But what we improved is more from a memory perspective. We want QMutex to use as little memory as possible, so you can put more QMutex in your objects to have more lock granularity.

sizeof(QMutex) == sizeof(void*).

That condition was already true in Qt 4. But Qt 4 has lots of hidden costs because it is using pimpl, as almost every class in Qt does. Constructing a QMutex in Qt 4 allocates and initializes a QMutexPrivate which itself initializes some platform specific primitives. So we have a memory overhead of over 120 bytes per mutex and also the cost of initialization/destruction. In Qt 5, there are no more hidden costs per mutex.

Another good point is that it is now a POD. This is good because QMutex is often used to lock global resources. Then a global mutex seems the obvious thing to use. But, in Qt, we want to avoid the use of global objects. Indeed, global objects should be avoided especially in library code, because the order of initialization is unspecified and they slow down the start-up even if that part of the library is not used. But as PODs do not need to be initialized, they can be used as global object.

QMutex singletonMutex; // static global object !!!  (OK only for POD)
MySingleton *MySingleton::self() {
    QMutexLocker locker(&singletonMutex)
    if(!m_self)
        m_self = new MySingleton;
    return m_self;
}

That code is dangerous in Qt 4 because maybe another global object in another compilation unit wants to make use of the singleton and the mutex could be used before it is created.

Summary of the QMutex changes from Qt 4.8 to Qt 5

• QMutex uses much less memory.
• The construction and destruction is faster.
• It is now suitable as a global object.
• The cost of locking or unlocking should not have changed.

Overview

class QMutex {
public:
    // API ...
private:
    QAtomicPointer<QMutexPrivate> d_ptr;
};

d_ptr is the only data member. The trick is that it is not always a pointer to an actual QMutexPrivate It may have a magic value:

You should be familiar with a pointer having 0 (or 0x0) as a value (also called NULL). 0x1 is a special value that represents an uncontended locked mutex.

So the principle of a lock is to try to atomically change the value of d_ptr from 0 to 1 If this succeeds, we have the lock, else, we need to wait.

bool QMutex::tryLock() {
    return d_ptr.testAndSetAcquire(0x0, 0x1);
}

void QMutex::lock() {
    while(!d_ptr.testAndSetAcquire(0x0, 0x1)) {
        // change the value of d_ptr to a QMutexPrivate and call wait on it.
        // ... see bellow
    }
}

The function bool QAtomicPointer::testAndSetAcquire(expected, newValue) will change the value of the pointer to newValue only if the previous value is expected and returns true if it succeeds. It does it atomically: If other threads change the value behind its back, the operation would fail and returns false.

Unlock

Let us start with the code:

void QMutex::unlock() {
    Q_ASSERT(d_ptr); // cannot be 0x0 because the mutex is locked.

    QMutexPrivate *d = d_ptr.fetchAndStoreRelease(0x0);
    if (d == 0x1)
        return; // no threads are waiting for the lock

    d->wake(); // wake up all threads
    d->deref();
}

The function fetchAndStoreRelease will atomically exchange the value of d_ptr to 0x0 and return the previous value. This exchange is atomic: it is guaranteed that we have the old value and that no threads had put a different value in between.

The returned value cannot be 0, because our mutex was locked (we have the lock, since we are unlocking it). If it was 0x1, no threads were waiting: we are finished. Otherwise, we have a pointer to QMutexPrivate and threads are waiting, so we need to wake all those threads up. They will then shortly try to lock the mutex again. Waking all the threads while only one can acquire the mutex is a waste of CPU. The real code inside Qt only wakes up one thread.

You will also notice a deref(). This will de-reference the QMutexPrivate and release it if needed.

Memory Management

In lock free programming, memory management is always a difficult topic. We use the technique of reference counting in order to make sure the QMutexPrivate is not released while one thread is trying to use it.

It is important to see that the QMutexPrivate can never be deleted. Consider this code:

d_ptr->ref().

This is not an atomic operation. The ref() itself is atomic, but there is a step before. The code first reads the value of d_ptr and puts the memory address in a register. But another thread may unlock the mutex and release the QMutexPrivate before the call to ref(). We would end up accessing free'ed memory.

QMutexPrivate are never deleted. Instead, they are put in a pool to be reused (a lock-free stack). We expect the total number of QMutexPrivate to be rather small. A QMutexPrivate is only needed if there is a thread waiting, and there cannot be more threads waiting than the number of threads on the application.

Here is how the QMutexPrivate looks

class QMutexPrivate {
    QAtomicInt refCount;
public:
    bool ref() {
        do {
            int c = refCount;
            if (c == 0)       // do not reference a QMutexPrivate that
                return false; //   has been already released.
        } while (!refCount.testAndSetAcquire(c, c + 1))
        return true;
    }
    void deref() {
        if (!refCount.deref())
            release();
    }

    /* Release this QMutexPrivate by pushing it on the
       global lock-free stack */
    void release();

    /* Pop a mutex private from the global lock-free stack
       (or allocate a new one if the stack was empty)
       And make sure it is properly initialized
       The refCount is initially 1
      */
    static QMutexPrivate *allocate();

    /* Block until wake is called.
       If wake was already called, returns immediately */
    void wait();

    /* Wake all the other waiting threads. All further calls to wait()
       will return immediately */
    void wake();

    // ... platform specifics
};

Lock

Now, we have got enough information to look at the implementation of the locking.

void QMutex::lock() {
    // Try to atomically change d_ptr from 0 to 1
    while (!d_ptr.testAndSetAcquire(0x0, 0x1)) {
        // If it did not succeed, we did not acquire the lock
        // so we need to wait.

        // We make a local copy of d_ptr to operate on a
        // pointer that does not change behind our back.
        QMutexPrivate *copy = d_ptr;

        // It is possible that d_ptr was changed to 0 before we did the copy.
        if (!copy)     // In that case, the mutex is unlocked,
            continue;  // so we can try to lock it again

        if (copy == 0x1) {
            // There is no QMutexPrivate yet, we need to allocate one
            copy = QMutexPrivate::allocate();
            if (!d_ptr.testAndSetOrdered(0x1, copy)) {
                // d_ptr was not 0x1 anymore, either the mutex was unlocked
                // or another thread has put a QMutexPrivate.
                // either way, release the now useless allocated QMutexPrivate,
                // and try again.
                copy->deref();
                continue;
            }
        }

        // Try to reference the QMutexPrivate
        if (!copy->ref())  // but if it fails because it was already released,
            continue;      // the mutex has been unlocked, so try again

        // Now, it is possible that the QMutexPrivate had been released,
        // but re-used again in another mutex. Hence, we need to check that
        // we are really holding a reference to the right mutex.
        if (d_ptr != copy) {
            copy->deref();
            continue;
        }

        // From this point we know that we have the right QMutexPrivate and that
        // it won't be released or change behind our back.
        // So we can wait.
        copy->wait();

        // The mutex has been unlocked, we can release the QMutexPrivate
        copy->deref();
        // and retry to lock.
    }
}

I hope that the comments were self explanatory.

To The Real Code.

The simplified algorithms code shown here has some limitations that have been solved in the real implementation inside Qt:

• QMutex has a recursive mode with the same API (the Qt 5 implementation supports them but they are more expensive than non-recursive mutex)
• This simplification wakes all the threads waiting on a single mutex instead of just one. That is not optimal since only one is going to acquire the lock.
• It is not fair. This means that a thread waiting for a long time might never acquire the mutex while two other threads always exchange the mutex between each other.
• There is the QMutex::tryLock(int timeout) that only blocks for a limited amount of time if the mutex cannot be acquired.

This code is also not used on Linux. On Linux, we use the futexes, using the d_ptr as the futex controller.

If you are interested, you can look at the actual code: the file qmutex.cpp, qmutex.h and qmutex_p.h available on Gitorious

This blog post is an introduction to lock-free programming. I'm writing this because this is the pre-requisite to understand my next post. This was also the content of my presentation for Qt Developer Days 2011.

Lock-free programming is the design of algorithms and data structures that do not acquire locks or mutexes.

When different threads in your program need to access the same data, we must ensure that the data is always in a coherent state when used. One way to achieve that is to do locking. A thread will acquire a mutex to write the data. That thread may touch the data structure and have it in an inconsistent state as it holds the mutex, but this is not a problem because other threads cannot access the data at this time as they will block waiting for the mutex to be released. While a thread is waiting, the OS will schedule another thread or process or let the CPU core rest.

What is wrong with Mutexes?

Mutexes are perfectly fine. But you have a problem if there is lock contention. If you want your algorithm to be fast, you want to use the available cores as much as possible instead of letting them sleep. A thread can hold a mutex and be de-scheduled by the CPU (because of a cache miss or its time slice is over). Then all the threads that want to acquire this mutex will be blocked. And if you have a lot of blocking, the OS also needs to do more context switches which are expensive because they clear the caches.

Other problems may arise if you do real time programming (priority inversion, convoying, ...). Mutexes also cannot be used in signal handlers.

Another example: Let us suppose you want to split your program into different processes so the whole application does not crash if one process crashes. (This is what modern browsers are doing by having the rendering of the page in a different process.) But if the process crashes while holding a shared lock you are in big trouble as this will most likely cause a dead lock in the main application.

So how can we do it without locking?

Modern CPUs have something called atomic operations. There are libraries that have APIs that let you use those atomic operations. Qt has two classes: QAtomicInt and QAtomicPointer. Other libraries or languages might have different primitives, but the principles are the same.

QAtomic API

I won't go into the detail of the API here as you can read the documentation of QAtomicInt and QAtomicPointer

But here are the highlights: Both classes have a similar API. They wrap an int or a pointer and allow to make atomic operations on it. There are 3 main operations: fetchAndAdd, fetchAndStore, and testAndSet. They are available in 4 variants, one for each ordering.

The one used here is testAndSet. It is also called Compare and Swap in the literature.
Here is a non-atomic implementation

bool QAtomicInt::testAndSet(int expectedValue, int newValue) {
    if (m_value != expectedValue)
        return false;
    m_value = newValue;
    return true;
}

What it does: it changes the wrapped value only if the value is the expected value, else it does not touch it and returns false.

It is atomic, meaning that if two threads operate at the value on the same time it stays consistent.

Of course, it is not implemented like this as it would not be atomic. It is implemented using assembly instructions. Qt's atomic classes are one of the very few places inside Qt implemented with assembly on each platform.

Memory Ordering

Today's CPUs have what is called out of order execution. What it means is that at each clock cycle, the CPU might read several instructions (say 6) from the memory and decode them and store them in a pool of instructions. The wires in the CPU will compute the dependencies between the instructions and feed the processing units with the instructions in the best possible order making the most efficient use of the CPU. So the instructions, and especially the reads and the store, are executed in an order that might be different from the one in the original program. The CPU is allowed to do that as long as it gives the same result on that thread.

However, we want to make sure that the ordering is preserved when we play with atomic operations. Indeed, if the memory is written in a different order, the data structure may be in an inconsistent state when other threads read the memory.

Here is an example:

  QAtomicPointer<int> p;
  int x;
  //...
  x = 4;
  p.fetchAndStoreRelease(&x);

It is important that when p is set to &x, the value of x is already 4. Else another thread could see the value of p that is still pointing to something else.

This is done by adding the proper memory fence in the program. Memory fences are special instructions that tell the CPU to not reorder. We have 4 kind of fences:

Acquire

No reads or writes that happen after will be moved before the atomic operations.

Release

The opposite of Acquire: No reads or writes are going to be moved after.

Ordered

It is a mix of the two previous orderings: Nothing can be moved after or before. This is the safest and the one to use if you don't know which one to use.

Relaxed

No memory fences are added.

The fence hints are added to the functions because on some architectures, there is one assembly instruction to do the atomic operation and the fence.

The fences are only there for keeping the CPU away from reordering. It has nothing to do with the fact that the compiler might also re-order everything. We make sure the compiler does not re-order by having 'volatile' access.

Lock-free Stack

We will here design a stack that works without locking:

class Stack {
    QAtomicPointer<Node> head;
public:
    Stack() : head(0) {}
    void push(Node *n) {
        do {
            n->next = head;
        } while(!head.testAndSetOrdered(n->next, n));
    }
    Node *pop() {
        Node *n;
        do {
            n = head;
            if (!n)
                return 0;
        } while(!head.testAndSetOrdered(n, n->next));
        return n;
    }
};

I'll use drawings to show how the code works:

It is basically implemented as a linked list: each node has a pointer to the next node, and we have a pointer to the first node called head.

Push

In this example, two threads want to push a node to the stack. Both threads have already executed the line n->next = head and will soon execute the atomic operation that will change head from the former head (B) to n (C or D)

In this image we see that the Thread 2 was first. And D is now on the stack.

The testAndSet in Thread 1 will fail. The head is not B anymore. head is not changed, meaning that the node D is still on the stack.

The Thread 1 will be notified that the testAndSet has failed and will then retry with the new head which is now D

Benchmark

So you can try yourself with this little example: lockfreestack.cpp
(Download the file in a new directory and do qmake -project && qmake && make)

This program first pushes 2 million nodes to the list using 4 threads, measuring the time it takes. Once all the threads have finished pushing, it will pop all those nodes using 4 threads and measure how long that takes

The program contains a version of the stack that uses QMutex (in the #if 0 block)

Results: (on my 4 core machine)

Not bad: the lock-free stack is more than 100 times faster. As you can see, there is much less contention (the real is smaller than the user) in the lock-free case, while with the mutex, a lot of time is spent blocking.

The ABA problem

OK, there is actually a big bug in our implementation. It works well in the benchmark because we push all the node then pop them all. There is no thread that pushes an pops nodes at the same time. In a real application, we might want to have a stack that works even if threads are pushing and popping nodes at the same time.

But what is the problem exactly? Again, I'll use some images to show:

In this example, the Thread 1 wants to pop a node. It take the address of A and will do a testAndSet to change head atomically from A to B. But it is de-scheduled by the OS just before the atomic operation while another thread is being executed

While Thread 1 is sleeping, Thread 2 also pops a node, so A is not on the stack anymore.

If Thread 1 wakes up now, the atomic operation in Thread 1 will fail because head is not anymore equal to A. But Thread 1 does not wake up and Thread 2 continues...

Thread 2 has pushed a node (C). Again, if thread 1 would wake up now, there would not be any problem, but it still does not wake up.

And Thread 2 pushes A back in the stack

But now, Thread 1 wakes up and execute the testAndSet, which succeeds as head is A again. This is a problem because now C is leaking.

It could have been even worse if Thread 2 had popped the node B.

Solutions to the ABA Problem

Every problem has solutions. It is outside the scope of this article to show the solutions in details. I would just give some hint that will orient your research on the web:

• Adding a serial number to the pointer, incremented each time a node is popped. It can be stored inside the least significant bits of the pointer (considering it points to an aligned address). But that might not be enough bits. Instead of using pointers, one can use indexes in array, leaving more bits for the serial number.
• Hazard pointer: Each thread puts the pointer it reads in a list readable by all threads. Those lists are then checked before reusing a node.
• Double or multiple word compare and swap. (which is possible using single word compare and swap)

Conclusions

As you might see, developing lock-free algorithm requires much more thinking than writing blocking algorithms. So keep mutexes, unless you have a lot of lock contention or are looking for a challenge.

A question I often got is whether it is not better to lock in order to let the other threads working instead of entering what might appear to be a spin lock. But the reality is that it is not a spin lock. The atomic operations succeed much more often than they fail. And they only fail if there was progress (that is, another thread made progress).

If you need help in your Qt applications regarding threads and locking, maybe Woboq can help you.