PyTorch入门及例程学习

本篇文章大部分内容翻译自learning pytorch with examples

1.PyTorch介绍

PyTorch是使用GPU和CPU优化的深度学习张量库，该项目2017年1月由facebook开源，短短两年时间，github上星数已经有25000+，增长速度非常快。

Pytorch的底层和Torch框架一样，但是使用Python重新写了很多内容，不仅更加灵活，支持动态图，而且提供了Python接口。是一个以Python优先的深度学习框架，PyTorch既可以看作加入了GPU支持的numpy，同时也可以看成一个拥有自动求导功能的强大的深度神经网络。除了Facebook外，它已经被Twitter、CMU和Salesforce等机构采用。最近看到大神贾扬清的一篇采访，说目前facebook一半以上的应用使用pytorch搭建，fb对此的投入还是很大的。

下面的代码参考文献2官方的example，说说深度学习与反向传播及python如何实现。输入batch为64，长度为1000的张量，经过FC + ReLU + FC，损失函数定义为MSE，输出batch为64，长度为10的张量。

2.Examples

（1）numpy实现

numpy是python进行数值计算的库，这个库中主要的处理都是由C和C++实现的，因此效率还是很高的。

import numpy as np
# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10
# Create random input and output data
x = np.random.randn(N, D_in) #64*1000
y = np.random.randn(N, D_out) #64*10
# Randomly initialize weights
w1 = np.random.randn(D_in, H) #1000*100
w2 = np.random.randn(H, D_out) #100*10
learning_rate = 1e-6
for t in range(500):
# Forward pass: compute predicted y
h = x.dot(w1) #same to np.dot(x,w1), h:64*100
h_relu = np.maximum(h, 0) #64*100
y_pred = h_relu.dot(w2) #64*10
# Compute and print loss
loss = np.square(y_pred - y).sum()
print(t, loss)
# Backprop to compute gradients of w1 and w2 with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_w2 = h_relu.T.dot(grad_y_pred) #h_relu.T is the transpose of h_relu，也就是转置,h_relu.T:100*64
grad_h_relu = grad_y_pred.dot(w2.T)
grad_h = grad_h_relu.copy()
grad_h[h <0] = 0
grad_w1 = x.T.dot(grad_h)
# Update weights
w1 -= learning_rate * grad_w1
w2 -= learning_rate * grad_w2

代码保存为test2.py，运行结果如下，收敛还是很快的。

（2）PyTorch实现

这里需安装pytorch，参考官方教程安装就好，我的环境使用anaconda来安装的，需要说明的是，pytorch支持cpu和gpu，通过torch.device可以很方便的切换。

Numpy是一个很好用的库，但它不能利用GPU来加速其数值计算。对于深度神经网络来说，GPU通常提供50倍或更高的加速。而使用PyTorch可以利用GPU来加速，Tensor(张量）是PyTorch中最基本的概念，其实就是一个n维数组，Pytorch提供了与Tensor相关的很多操作，并且都可以在GPU上实现。

与numpy不同，PyTorch Tensors可以利用GPU加速其数值计算。要在GPU上运行PyTorch Tensor，只需将其转换为新的数据类型即可。

下面的代码将实现与上面NumPy相同的功能。

import torch
dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU
# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10
# Create random Tensors to hold input and outputs.
# Setting requires_grad=False indicates that we do not need to compute gradients
# with respect to these Tensors during the backward pass.
x = torch.randn(N, D_in, device=device, dtype=dtype)
y = torch.randn(N, D_out, device=device, dtype=dtype)
# Create random Tensors for weights.
# Setting requires_grad=True indicates that we want to compute gradients with
# respect to these Tensors during the backward pass.
w1 = torch.randn(D_in, H, device=device, dtype=dtype, requires_grad=True)
w2 = torch.randn(H, D_out, device=device, dtype=dtype, requires_grad=True)
learning_rate = 1e-6
for t in range(500):
# Forward pass: compute predicted y using operations on Tensors; these
# are exactly the same operations we used to compute the forward pass using
# Tensors, but we do not need to keep references to intermediate values since
# we are not implementing the backward pass by hand.
y_pred = x.mm(w1).clamp(min=0).mm(w2)
# Compute and print loss using operations on Tensors.
# Now loss is a Tensor of shape (1,)
# loss.item() gets the a scalar value held in the loss.
loss = (y_pred - y).pow(2).sum()
print(t, loss.item())
# Use autograd to compute the backward pass. This call will compute the
# gradient of loss with respect to all Tensors with requires_grad=True.
# After this call w1.grad and w2.grad will be Tensors holding the gradient
# of the loss with respect to w1 and w2 respectively.
loss.backward()
# Manually update weights using gradient descent. Wrap in torch.no_grad()
# because weights have requires_grad=True, but we don't need to track this
# in autograd.
# An alternative way is to operate on weight.data and weight.grad.data.
# Recall that tensor.data gives a tensor that shares the storage with
# tensor, but doesn't track history.
# You can also use torch.optim.SGD to achieve this.
with torch.no_grad():
w1 -= learning_rate * w1.grad
w2 -= learning_rate * w2.grad
# Manually zero the gradients after updating weights
w1.grad.zero_()
w2.grad.zero_()

（3）PyTorch AutoGrad实现

在前面的介绍中，我们需要手动实现前向网络forward和反向网络backward，对两层网络来说，很好实现，但对于分类、检测、分割等比较复杂的深度学习网络来说，就显得不够优雅了。

在PyTorch中，我们可以使用AutoGrad包，利用automatic differentation方法来自动计算backward，使用AutoGrad包时，前向网络定义了一套计算图，使用计算图来执行，计算图中的每个节点都定义为Tensors，通过这个计算图反向传播可以比较容易计算得到梯度值。

也就是说，如果x是一个张量，并且x.requires_grad=True，那么x.grad是对应x的梯度值的张量，下面来看利用AutoGrad库实现的两层网络。

import torch
dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU
# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10
# Create random Tensors to hold input and outputs.
# Setting requires_grad=False indicates that we do not need to compute gradients
# with respect to these Tensors during the backward pass.
x = torch.randn(N, D_in, device=device, dtype=dtype)
y = torch.randn(N, D_out, device=device, dtype=dtype)
# Create random Tensors for weights.
# Setting requires_grad=True indicates that we want to compute gradients with
# respect to these Tensors during the backward pass.
w1 = torch.randn(D_in, H, device=device, dtype=dtype, requires_grad=True)
w2 = torch.randn(H, D_out, device=device, dtype=dtype, requires_grad=True)
learning_rate = 1e-6
for t in range(500):
# Forward pass: compute predicted y using operations on Tensors; these
# are exactly the same operations we used to compute the forward pass using
# Tensors, but we do not need to keep references to intermediate values since
# we are not implementing the backward pass by hand.
y_pred = x.mm(w1).clamp(min=0).mm(w2)
# Compute and print loss using operations on Tensors.
# Now loss is a Tensor of shape (1,)
# loss.item() gets the a scalar value held in the loss.
loss = (y_pred - y).pow(2).sum()
print(t, loss.item())
# Use autograd to compute the backward pass. This call will compute the
# gradient of loss with respect to all Tensors with requires_grad=True.
# After this call w1.grad and w2.grad will be Tensors holding the gradient
# of the loss with respect to w1 and w2 respectively.
loss.backward()
# Manually update weights using gradient descent. Wrap in torch.no_grad()
# because weights have requires_grad=True, but we don't need to track this
# in autograd.
# An alternative way is to operate on weight.data and weight.grad.data.
# Recall that tensor.data gives a tensor that shares the storage with
# tensor, but doesn't track history.
# You can also use torch.optim.SGD to achieve this.
with torch.no_grad():
w1 -= learning_rate * w1.grad
w2 -= learning_rate * w2.grad
# Manually zero the gradients after updating weights
w1.grad.zero_()
w2.grad.zero_()

非常的简单就实现了两层网络。

至繁归于至简！

参考：

[1] https://github.com/pytorch/pytorch

[2] https://pytorch.org/tutorials/beginner/pytorch_with_examples.html

[3] https://baijiahao.baidu.com/s?id=1590200756011465121&wfr=spider&for=pc