【笔记】训练神经网络时解决显存不足的办法：核常兼备，常：减少参数规模；核：减少参数大小

常规方法1&＃xff1a;

 常规方法2&＃xff1a;

GitHub - gpleiss/efficient_densenet_pytorch: A memory-efficient implementation of DenseNets
model2GitHub - mapillary/inplace_abn: In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

删除loss&＃xff1a; Tensor to Variable and memory freeing best practices - #2 by fmassa - PyTorch Forums

降低精度&＃xff1a;

https://github.com/NVIDIA/apex

分清楚 eval 和 requires_grad &＃61; False

&＃39;model.eval()&＃39; vs &＃39;with torch.no_grad()&＃39; - PyTorch Forums

使用torch.cuda.empty_cache()&＃xff0c;在确定的地方释放显存

How can we release GPU memory cache? - PyTorch Forums

核子武器&＃xff1a;

    参考资料&＃xff1a;

    {
        [1] https://discuss.pytorch.org/t/cuda-memory-continuously-increases-when-net-images-called-in-every-iteration/501

        [2] https://forums.fast.ai/t/clearing-gpu-memory-pytorch/14637/3

        [3] https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management

        [4] https://discuss.pytorch.org/t/how-to-debug-causes-of-gpu-memory-leaks/6741/3

    }   

}