PyTorch Developer Podcast

Inference mode

Episode Summary

What's inference mode? Why doesn't my code run fast if I use no_grad or make sure requires_grad=False? How come inference mode is safe but AutoNonVariableTypeMode is not?

Episode Notes

What's inference mode? Why doesn't my code run fast if I use no_grad or make sure requires_grad=False? How come inference mode is safe but AutoNonVariableTypeMode is not?

Further reading:

• Inference mode RFC https://github.com/.../rfc0011/RFC-0011-InferenceMode.md
• Inference mode docs for C++ frontend users https://github.com/.../cpp/source/notes/inference_mode.rst
• Tracking issue for Python frontend support https://github.com/pytorch/pytorch/issues/56608