How Qt Signals and Slots Work - Part 3 - Queued and Inter Thread Connections

In this article, we will explore the mechanisms powering the Qt queued connections.

Summary from Part 1

In the first part, we saw that signals are just simple functions, whose body is generated by moc. They are just calling QMetaObject::activate, with an array of pointers to arguments on the stack. Here is the code of a signal, as generated by moc: (from part 1)

// SIGNAL 0
void Counter::valueChanged(int _t1)
{
    void *_a[] = { Q_NULLPTR, const_cast<void*>(reinterpret_cast<const void*>(&_t1)) };
    QMetaObject::activate(this, &staticMetaObject, 0, _a);
}

QMetaObject::activate will then look in internal data structures to find out what are the slots connected to that signal. As seen in part 1, for each slot, the following code will be executed:

// Determine if this connection should be sent immediately or
// put into the event queue
if ((c->connectionType == Qt::AutoConnection && !receiverInSameThread)
        || (c->connectionType == Qt::QueuedConnection)) {
    queued_activate(sender, signal_index, c, argv, locker);
    continue;
} else if (c->connectionType == Qt::BlockingQueuedConnection) {
    /* ... Skipped ... */
    continue;
}
/* ... DirectConnection: call the slot as seen in Part 1 */

So in this blog post we will see what exactly happens in queued_activate and other parts that were skipped for the BlockingQueuedConnection

Qt Event Loop

A QueuedConnection will post an event to the event loop to eventually be handled.

When posting an event (in QCoreApplication::postEvent), the event will be pushed in a per-thread queue (QThreadData::postEventList). The event queued is protected by a mutex, so there is no race conditions when threads push events to another thread's event queue.

Once the event has been added to the queue, and if the receiver is living in another thread, we notify the event dispatcher of that thread by calling QAbstractEventDispatcher::wakeUp. This will wake up the dispatcher if it was sleeping while waiting for more events. If the receiver is in the same thread, the event will be processed later, as the event loop iterates.

The event will be deleted right after being processed in the thread that processes it.

An event posted using a QueuedConnection is a QMetaCallEvent. When processed, that event will call the slot the same way we call them for direct connections. All the information (slot to call, parameter values, ...) are stored inside the event.

Copying the parameters

The argv coming from the signal is an array of pointers to the arguments. The problem is that these pointers point to the stack of the signal where the arguments are. Once the signal returns, they will not be valid anymore. So we'll have to copy the parameter values of the function on the heap. In order to do that, we just ask QMetaType. We have seen in the QMetaType article that QMetaType::create has the ability to copy any type knowing it's QMetaType ID and a pointer to the type.

To know the QMetaType ID of a particular parameter, we will look in the QMetaObject, which contains the name of all the types. We will then be able to look up the particular type in the QMetaType database.

queued_activate

We can now put it all together and read through the code of queued_activate, which is called by QMetaObject::activate to prepare a Qt::QueuedConnection slot call. The code showed here has been slightly simplified and commented:

static void queued_activate(QObject *sender, int signal,
                            QObjectPrivate::Connection *c, void **argv,
                            QMutexLocker &locker)
{
  const int *argumentTypes = c->argumentTypes;
  // c->argumentTypes is an array of int containing the argument types.
  // It might have been initialized in the connection statement when using the
  // new syntax, but usually it is `nullptr` until the first queued activation
  // of that connection.

  // DIRECT_CONNECTION_ONLY is a dummy int which means that there was an error
  // fetching the type ID of the arguments.

  if (!argumentTypes) {
    // Ask the QMetaObject for the parameter names, and use the QMetaType
    // system to look up type IDs
    QMetaMethod m = QMetaObjectPrivate::signal(sender->metaObject(), signal);
    argumentTypes = queuedConnectionTypes(m.parameterTypes());
    if (!argumentTypes) // Cannot queue arguments
      argumentTypes = &DIRECT_CONNECTION_ONLY;
    c->argumentTypes = argumentTypes; /* ... skipped: atomic update ... */
  }
  if (argumentTypes == &DIRECT_CONNECTION_ONLY) // Cannot activate
      return;
  int nargs = 1; // Include the return type
  while (argumentTypes[nargs-1])
      ++nargs;
  // Copy the argumentTypes array since the event is going to take ownership
  int *types = (int *) malloc(nargs*sizeof(int));
  void **args = (void **) malloc(nargs*sizeof(void *));

  // Ignore the return value as it makes no sense in a queued connection
  types[0] = 0; // Return type
  args[0] = 0; // Return value

  if (nargs > 1) {
    for (int n = 1; n < nargs; ++n)
      types[n] = argumentTypes[n-1];

    // We must unlock the object's signal mutex while calling the copy
    // constructors of the arguments as they might re-enter and cause a deadlock
    locker.unlock();
    for (int n = 1; n < nargs; ++n)
      args[n] = QMetaType::create(types[n], argv[n]);
    locker.relock();

    if (!c->receiver) {
      // We have been disconnected while the mutex was unlocked
      /* ... skipped cleanup ... */
      return;
    }
  }

  // Post an event
  QMetaCallEvent *ev = c->isSlotObject ?
    new QMetaCallEvent(c->slotObj, sender, signal, nargs, types, args) :
    new QMetaCallEvent(c->method_offset, c->method_relative, c->callFunction,
                       sender, signal, nargs, types, args);
  QCoreApplication::postEvent(c->receiver, ev);
}

Upon reception of this event, QObject::event will set the sender and call QMetaCallEvent::placeMetaCall. That later function will dispatch just the same way as QMetaObject::activate would do it for direct connections, as seen in Part 1

  case QEvent::MetaCall:
  {
    QMetaCallEvent *mce = static_cast<QMetaCallEvent*>(e);

    QConnectionSenderSwitcher sw(this, const_cast<QObject*>(mce->sender()),
                                 mce->signalId());

    mce->placeMetaCall(this);
    break;
  }

BlockingQueuedConnection

BlockingQueuedConnection is a mix between DirectConnection and QueuedConnection. Like with a DirectConnection, the arguments can stay on the stack since the stack is on the thread that is blocked. No need to copy the arguments. Like with a QueuedConnection, an event is posted to the other thread's event loop. The event also contains a pointer to a QSemaphore. The thread that delivers the event will release the semaphore right after the slot has been called. Meanwhile, the thread that called the signal will acquire the semaphore in order to wait until the event is processed.

} else if (c->connectionType == Qt::BlockingQueuedConnection) {
  locker.unlock(); // unlock the QObject's signal mutex.
  if (receiverInSameThread) {
    qWarning("Qt: Dead lock detected while activating a BlockingQueuedConnection: "
             "Sender is %s(%p), receiver is %s(%p)",
             sender->metaObject()->className(), sender,
             receiver->metaObject()->className(), receiver);
  }
  QSemaphore semaphore;
  QMetaCallEvent *ev = c->isSlotObject ?
    new QMetaCallEvent(c->slotObj, sender, signal_index, 0, 0, argv, &semaphore) :
    new QMetaCallEvent(c->method_offset, c->method_relative, c->callFunction,
                       sender, signal_index, 0, 0, argv , &semaphore);
  QCoreApplication::postEvent(receiver, ev);
  semaphore.acquire();
  locker.relock();
  continue;
}

It is the destructor of QMetaCallEvent which will release the semaphore. This is good because the event will be deleted right after it is delivered (i.e. the slot has been called) but also when the event is not delivered (e.g. because the receiving object was deleted).

A BlockingQueuedConnection can be useful to do thread communication when you want to invoke a function in another thread and wait for the answer before it is finished. However, it must be done with care.

The dangers of BlockingQueuedConnection

You must be careful in order to avoid deadlocks.

Obviously, if you connect two objects using BlockingQueuedConnection living on the same thread, you will deadlock immediately. You are sending an event to the sender's own thread and then are locking the thread waiting for the event to be processed. Since the thread is blocked, the event will never be processed and the thread will be blocked forever. Qt detects this at run time and prints a warning, but does not attempt to fix the problem for you. It has been suggested that Qt could then just do a normal DirectConnection if both objects are in the same thread. But we choose not to because BlockingQueuedConnection is something that can only be used if you know what you are doing: You must know from which thread to what other thread the event will be sent.

The real danger is that you must keep your design such that if in your application, you do a BlockingQueuedConnection from thread A to thread B, thread B must never wait for thread A, or you will have a deadlock again.

When emitting the signal or calling QMetaObject::invokeMethod(), you must not have any mutex locked that thread B might also try locking.

A problem will typically appear when you need to terminate a thread using a BlockingQueuedConnection, for example in this pseudo code:

void MyOperation::stop()
{
    m_thread->quit();
    m_thread->wait(); // Waits for the callee thread, might deadlock
    cleanup();
}

// Connected via a BlockingQueuedConnection
Stuff MyOperation::slotGetSomeData(const Key &k)
{
    return m_data->get(k);
}

You cannot just call wait here because the child thread might have already emitted, or is about to emit the signal that will wait for the parent thread, which won't go back to its event loop. All the thread cleanup information transfer must only happen with events posted between threads, without using wait(). A better way to do it would be:

void MyOperation::stop()
{
    connect(m_thread, &QThread::finished, this, &MyOperation::cleanup);
    m_thread->quit();
    /* (note that we connected before calling quit to avoid a race) */
}

The downside is that MyOperation::cleanup() is now called asynchronously, which may complicate the design.

Conclusion

This article should conclude the series. I hope these articles have demystified signals and slots, and that knowing a bit how this works under the hood will help you make better use of them in your applications.