1. Tensor order for training images, [N, C, H, W]
2. Pytorch not returning gpu memory ( https://github.com/pytorch/pytorch/issues/1085 )
github replies from Adam Paszke :
pkill -9 python
try pkill -9 python in terminal, it should kill all python processes and free their resources.