RyanKaskel.com

TL; DR Don’t use a string literal as the value of the sender attribute when defining your own Django signals.

I got bit by a nasty bug today using Django’s signal framework. I used the signal like so:

show_user_widget_responses = show_user_widget.send(
    sender='my_widget_view',
    user=user
)

I ran the tests to make sure the signal was being sent to the listeners (i.e receivers) and they all passed. I then noticed the view sent another signal in certain conditions and its sender was a set to a different string. For the sake of consistency, I changed the above signal’s sender string to “widget-user-view” and went to make a cup of tea.

When I returned to my desk, I got a bit distracted and continued to work on the feature. I ran the app’s tests and they started failing: the test listeners weren’t being called. Oops.

My first inclination was that I forgot to change the sender argument in “show_user_widget.connect.” Nope… the tests were still failing. I decided to change the sender argument in the send call to “widget” and the tests started passing again. What was going on? I proceeded to try new sender values and I soon came to believe that hyphens weren’t allowed in sender arguments that were strings. When I start thinking like this, I know that it’s time to take a step back and use Google.

One of the first hits I came across was ticket #16447: settings_changed signal uses strings for sender, filed by Alex Gaynor. He noticed that when he ran Django’s tests using PyPy, the test that checked the “settings_changed” signal started failing. This was exactly my problem, so what was going on?

It turns out that Django’s signal dispatcher code uses the id() of the sender argument as part of the “key” to the receiver (which I’ve always called a “listener”).  Here is the relevant code in django.dispatch.dispatcher:

def _make_id(target):
    if hasattr(target, 'im_func'):
        return (id(target.im_self), id(target.im_func))
    return id(target)

Here is how the key is defined if “dispatch_uid” is not set (in Signal.connect):

lookup_key = (_make_id(receiver), _make_id(sender))

This is the core of Signal.send:

for receiver in self._live_receivers(_make_id(sender)):
    response = receiver(signal=self, sender=sender, **named)
    responses.append((receiver, response))

And finally, here is the important section of Signal._live_receivers:

def _live_receivers(self, senderkey)
    ...
    for (receiverkey, r_senderkey), receiver in self.receivers:
        if r_senderkey == none_senderkey or r_senderkey == senderkey:
        ...
    ...

Once this implementation detail is known, it becomes obvious that this is a manifestation of the difference between identity and equality: id(“my-string”) is not guaranteed to always return the same integer even though “my-string” == “my-string”. Because I used a string literal as the argument, new string objects with potentially different identities were being created in each call. I don’t know exactly why their identity differs, but the behavior is easily observable in the shell:

>>> id('widget-user-view')
37951112
>>> id('widget-user-view')
37951232
>>> id('widget-user-view')
37951272
>>> id('widget-user-view')
37951112
>>> id('widget-user-view')
37951232

But… certain strings, like my original value of sender, “my_widget_view,” will return the same integer ID:

>>> id('my_widget_view')
37951192
>>> id('my_widget_view')
37951192
>>> id('my_widget_view')
37951192
>>> id('my_widget_view')
37951192

So, the moral of the story is: don’t use string literals for the sender argument when implementing custom signals in Django. It turns out to be tricky to debug. If you’re sending a signal from a class and want a good value for sender, try “self.__class__.”  As Justin pointed out in a comment, you can use a string for the value of sender if you use that same string variable as the argument in each call to the signal methods (you must import the string along with the signal).

I still think it’s better to avoid strings as senders altogether. The ticket filed by Alex Gaynor referenced above used a setting name as the sender and it wouldn’t be long before someone tried:

signals.settings_changed.connect(listener, sender="SOME_SETTING", ...)

rather than the more correct

from django.conf import settings
signals.settings_changed.connect(listener, sender=settings.SOME_SETTING, ...)

It’s a subtle bug waiting to happen and is indeed why the test failed for Alex. Furthermore, you also may not even need to pass a value for sender, if, for example, you only use Signal.send in one place or if the listener (receiver) doesn’t need to use it. (see the code reference above: _make_id(None) is also checked).

Another mistake I made in my code is now obvious: always use “constants” (i.e. BIG_BOLD_VARIABLES defined at the top of the module) instead of peppering the same string literal around your code.

I later found another ticket that describes the same problem. One core dev marked it as “wontfix” and I agree that there is nothing to fix in the code. But given that another core dev made the same mistake in changeset 16327, I think this behavior should be documented and the next time I get some free time, I will submit a documentation patch (after I write my dissertation .