tensorflowLSTM

Note that if we wrap a base cell with dropout and then use it to build a MultiRNNCell, both input dropout and output dropout will be applied between layers (so if both are, say, 0.9, the dropout in between layers will be 0.9 * 0.9 = 0.81). If we want equal dropout on all inputs and outputs of a multi-layered RNN, we can use only output or input dropout on the base cell, and then wrap the entire MultiRNNCell with the input or output dropout like so:

cell = tf.nn.rnn_cell.LSTMCell(state_size, state_is_tuple=True)
cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=global_dropout)
cell = tf.nn.rnn_cell.MultiRNNCell([cell] * num_layers, state_is_tuple=True)
cell = tf.nn.rnn_cell.DropoutWrapper(cell, output_keep_prob=global_dropout)

LSTMCell和BasicLSTMCell的区别：
1. 增加了use_peepholes, bool值，为True时增加窥视孔。图片来源于：http://colah.github.io/posts/2015-08-Understanding-LSTMs/

2. 增加了cell_clip, 浮点值，把cell的值限制在 ±cell_clip内

c = clip_ops.clip_by_value(c, -self._cell_clip, self._cell_clip)

3. 增加了num_proj（int）和proj_clip(float), 相对于BasicLSTMCell，在输出m计算完之后增加了一层线性变换，并限制了输出的值

m = _linear(m, self._num_proj, bias=False, scope=scope)
m = clip_ops.clip_by_value(m, -self._proj_clip, self._proj_clip)