TL;DR : Use this script to measure sizes of objects in python. This works for iterables, dicts and subclasses of Python’s built-in Object class .
A few months ago I suspected a global variable to be the root cause of a memory leak. I had enough empirical proof to remove it from our production systems, however not enough to satisfy my inner nerd.
To do that I had to measure the object’s size in memory, which, surprisingly is not a straightforward thing to do in Python. Taking a look at the top StackOverflow answers for “measuring python object sizes” wasn’t satisfying as they only pointed to large chunks of code that I’d have to use to instrument my code and they were way overkill.
What made this worse was the confusing Python docs about sys.getsizeof :
Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.
From an OOP standpoint, this is not true. When you measure a size of an object, you really want the size of all of it’s attributes, and their attributes, etc. sys.getsizeof only gives you the size of the object and their attributes, however it does not recursively add the size of sub-attributes.
So I decided to fill in the gap. I wrote the helper below to recursively measure the size of a Python object (or dict, tuple, list etc). It uses sys.getsizeof internally and it turned out to be pretty easy to write once I figured out enough of the edge cases.
def get_size(obj):"""Recursively finds size of objects"""
size = 0
if isinstance(obj, dict):
size += sum([get_size(v) for v in obj.values()])
size += sum([get_size(k) for k in obj.keys()])
elif hasattr(obj, '__dict__'):
size += get_size(obj.__dict__)
elif hasattr(obj, '__iter__'):
size += sum([get_size(i) for i in obj])
else:
size += sys.getsizeof(obj)
return size
I hope it helps the next time you need to accurately measure the size of an object!
BTW, we're hiring at Shippo! We workwithPython,EmberJS, and PostgreSQL.Our API shipsmillions of packages around the world.