PyTorch Developer Podcast

History and constraints of Tensor

Episode Summary

What historical constraints and design choices lead to the design of Tensor/Storage (and their Impl variants) as they are today? Why do we use intrusive refcounting? Why are we trying to get rid of virtual methods on TensorImpl? Why are there so many frickin' bitfields?

Episode Notes

What historical constraints and design choices lead to the design of Tensor/Storage (and their Impl variants) as they are today? Why do we use intrusive refcounting? Why are we trying to get rid of virtual methods on TensorImpl? Why are there so many frickin' bitfields?

Further reading.

• PyTorch internals blog post http://blog.ezyang.com/2019/05/pytorch-internals/
• Writing Python in C++, a manifesto https://github.com/pytorch/pytorch/wiki/Writing-Python-in-cpp-(a-manifesto)
• At time of writing, the breakdown of all fields on TensorImpl https://github.com/pytorch/pytorch/blob/71f4c5c1f436258adc303b710efb3f41b2d50c4e/c10/core/TensorImpl.h#L2155-L2177