I’ve been building an application that uses Python’s datetime.date class to represent financial quarters. The system hardcodes the month and day of each quarter, so the first quarter, for example, is equal to March 31st, and does not vary year to year.

Since these month and day combinations are already hardcoded, I decided to subclass datetime.date to better handle quarters throughout the system. The process was more complicated than I expected.

The subclass started simply:

import datetime

class Quarter(datetime.date):
    pass

Since it inherited the functionality of date, this class was ready to use:

q1 = Quarter(2018, 3, 31)

print(type(q1)) # <class '__main__.Quarter'>
print(repr(q1)) # Quarter(2018, 3, 31)
print(q1) # 2018-03-31

But I had yet to improve upon date. To make this class useful for my application, I wanted a different constructor. Since each quarter has a corresponding month and day, I just wanted to pass a year and quarter, such as Quarter(2018, 1).

Knowing the basics of Python classes, I started by overriding the initialization function:

_q1 = (3, 31)
_q2 = (6, 30)
_q3 = (9, 30)
_q4 = (12, 31)

_qs = [_q1, _q2, _q3, _q4]

class Quarter(datetime.date):
    def __init__(self, year, q):
        if q < 1 or q > 4:
            raise ValueError("Quarter must be 1, 2, 3 or 4")
        month, day = _qs[q-1]
        super().__init__(year, month, day)

But when attempting to call the constructor Quarter(2018, 1), I received the following error:

TypeError: Required argument 'day' (pos 3) not found

What’s going on here? I thought I had removed the day argument? Did I screw up the initialization of the super type?

Stepping through the debugger, I discovered that the __init__ function is never called, and the above error originates from a function called __new__. This function is present on immutable types, which includes all types in the datetime package.

The __new__ function is not mentioned anywhere in the Python tutorial on classes. The best description I could find is in the data model reference. In brief, __new__ is called to create a new instance of a class, which will then be initialized.

The function __new__ is a static method, though a special case since it does not need to be declared as one. It takes cls as its first argument instead of self and returns an instance of our desired class. Just as I did with my first attempt at initialization, I wanted to use super() and trust the behavior of date.__new__:

class Quarter(datetime.date):
    def __new__(cls, year, q):
        if q < 1 or q > 4:
            raise ValueError("Quarter must be 1, 2, 3 or 4")
        month, day = _qs[q-1]
        return super().__new__(cls, year, month, day)

It worked! And the type() and repr() outputs were the same as before!

But wait, the repr() output was now a problem, since the constructor changed. Calling eval(repr(Quarter(2018, 1))) with the above code throws:

TypeError: __new__() takes 3 positional arguments but 4 were given

I needed to override the inherited __repr__ magic method and return a value appropriate for the new constructor (if you’re unfamiliar with __repr__ you can read more here). Since doing so required the instance’s q value, which I suspected would be used frequently, I added it as a computed property:

class Quarter(datetime.date):
    def __new__(cls, year, q):
        if q < 1 or q > 4:
            raise ValueError("Quarter must be 1, 2, 3 or 4")
        month, day = _qs[q-1]
        return super().__new__(cls, year, month, day)

    @property
    def q(self):
        return self.month // 3 # We want an int type

    def __repr__(self):
        return '{0}({1}, {2})'.format(self.__class__.__name__, self.year, self.q)

Wth the eval issue fixed, I started using the new subclass throughout the system. But there was still a significant problem with the above code, and I only discovered it when pickling:

q1 = Quarter(2018, 1)
pickle.loads(pickle.dumps(q1))

Which throws:

TypeError: new() missing 1 required positional argument: 'q'

Another constructor issue!

Finding and fixing issues such as the above would be easier with the datetime package source code, but since the major implementation of Python is cpython, you’ll need to know some C to read its source.

There is also a Python implementation in pure Python called pypy, but we cannot guarantee that its implementation is equivalent to cpython.

Returning to the issue with pickle, I attempted to find the cause of the TypeError. The pickle documentation lists six magic methods that can all alter the behavior of pickle; some take precedence over others, and most have a variety of return types. There are also five different pickle protocols in use as of Python 3.4, which affect the usage of these methods. Additionally, four of these methods aren’t even unique to pickle, and are instead part of the copy protocol.

So which methods did I need to implement to fix the pickle (and copy) behavior the subclass? Since the pickle documentation explicitly warned against writing your own __reduce__, calling it “error prone”, I started with __getnewargs__, since it controls arguments passed to __new__:

class Quarter(datetime.date):
    # Other methods omitted...
    def __getnewargs__(self):
        return (self.year, self.q)

But when re-trying pickle, I received the same TypeError as before. And when stepping through the code, I discovered that __getnewargs__ is never called!

I forgot that I was writing a subclass! Looking through the cpython source, I found that date has a __reduce__ method the uses a private getstate method (which is different from the __getstate__ magic method). According to the pickle documentation, the __reduce__ method takes precedence over __getnewargs__, so it is never called!

Thankfully, I had found the solution to the error - re-implement __reduce__ despite the documentation’s warning:

class Quarter(datetime.date):
    # Other methods omitted...
    def __reduce__(self):
        return (self.__class__, (self.year, self.q))

I could now safely copy and pickle the subclass:

q1 = Quarter(2018, 1)
copy.copy(q1)
copy.deepcopy(q1)
pickle.loads(pickle.dumps(q1))

I also added other magic methods to customize the subclass. I only needed a few, since the subclass inherits existing features of date, including comparisons and hashing.

Here’s a gist of the complete Quarter class, which shows off some of this inherited behavior.

Overall, I was disappointed by the complexity of subclassing a built-in, immutable type.