Exploration of the blog “Why Python is Slow: Looking Under the Hood”

“Why Python is Slow: Looking Under the Hood” is an interesting blog can be found here. In this article, I’d like to reproduce some interesting observations.

Number of reference for integers

The original code from [1]

import matplotlib.pyplot as plt                                                         
import sys                                                                              
def plot():                                                                             
    plt.loglog(range(1000), [sys.getrefcount(i) for i in range(1000)])                  
    plt.xlabel('integer value')                                                         
    plt.ylabel('reference count')                                                       
    plt.show()                                                                          
                                                                                        
if __name__ == "__main__":                                                              
    plot()                                                                              

Plot of above code

We can also slightly change the above code to see more details of value < 300

import matplotlib.pyplot as plt                                                         
import sys                                                                              
def plot(n):                                                                            
    plt.plot(range(n), [sys.getrefcount(i) for i in range(n)])                          
    plt.xlabel('integer value')                                                         
    plt.ylabel('reference count')                                                       
    plt.show()                                                                          
                                                                                        
if __name__ == "__main__":                                                              
    plot(300)     

There is a peak for the number of 256, this should be the threshold used by Cython. Original, I though it should be 255, it is interesting to see that it is actually 256.

[(252, 6), (253, 6), (254, 10), (255, 52), (256, 368), (257, 3)]

From the post here, the integer is currently from -5 to 256.

An dangerous experiment

The following code is slightly adjusted from [1]

import ctypes                                                                           
                                                                                        
class IntStruct(ctypes.Structure):                                                      
    _fields_ = [("ob_refcnt", ctypes.c_long),                                           
                ("ob_type", ctypes.c_void_p),                                           
                ("ob_size", ctypes.c_ulong),                                            
                ("ob_digit", ctypes.c_long)]                                            
                                                                                        
    def __repr__(self):                                                                 
        return ("IntStruct(ob_digit={self.ob_digit}, "                                  
                "refcount={self.ob_refcnt})").format(self=self)                         
                                                                                        
                                                                                        
def dangerous_experiment():                                                             
    raw_input = input("This experiment is dangerous, are you sure you wanna run? Type:'yes, I am sure!' to run the code! \n")
    assert raw_input == "yes, I am sure!", "Seems you are not very sure! Quit!"         
    id113 = id(113)                                                                     
    iptr = IntStruct.from_address(id113)                                                
    iptr.ob_digit = 4  # now Python's 113 contains a 4!                                 
    print(f"113 == 4 {113 == 4}")                                                       
                                                                                        
                                                                                        
if __name__ == "__main__":                                                              
    dangerous_experiment()   

Output

This experiment is dangerous, are you sure you wanna run? Type:'yes, I am sure!' to run the code! 
yes, I am sure!
113 == 4 True

No worries, those code only have effect during the running. DO NOT PUT THE CODE TO production. The python version used to test the above code

Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:50:38) 
[Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 113 == 4
False

Reference

[1] Why Python is Slow: Looking Under the Hood