文章目录



微信公众号

阅读完《TensorFlow 学习摘要(三) 深度学习 - 学习率》文章后,我们现在可以对之前《TensorFlow 学习摘要(三) 深度学习 - 手写数字识别入门04 - Dropout 方法优化》中的案例进行改造,学习率指数衰减法再来看下结果。

添加代码,如下所示。

global_step = tf.Variable(0) 

learning_rate = tf.train.exponential_decay(0.01, global_step, 100, 0.95, staircase=True) 
train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss,global_step=global_step)

for epoch in range(20):
        for i in range(1000):
            global_step = i
            // ......

至此,我们优化完毕。现在,我们来看下完整的代码。

import tensorflow as tf

# 导入 input_data 用于自动下载和安装MNIST数据集
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# 放置占位符,用于在计算时接收输入值
x = tf.placeholder(tf.float32, [None, 784])
# 为了进行训练,需要把正确值一并传入网络
y_ = tf.placeholder(tf.float32, [None,10])

# 概率值
keep_prob = 0.5

# 创建深度神经网络
W1 = tf.Variable(tf.truncated_normal([784, 2000], stddev=0.1))
b1 = tf.Variable(tf.zeros([2000]) +0.1)
L1 = tf.nn.tanh(tf.matmul(x, W1) + b1)
L1_drop = tf.nn.dropout(L1, keep_prob)

W2 = tf.Variable(tf.truncated_normal([2000, 2000], stddev=0.1))
b2 = tf.Variable(tf.zeros([2000]) +0.1)
L2 = tf.nn.tanh(tf.matmul(L1_drop, W2) + b2)
L2_drop = tf.nn.dropout(L2, keep_prob)

W3 = tf.Variable(tf.truncated_normal([2000, 1000], stddev=0.1))
b3 = tf.Variable(tf.zeros([1000]) +0.1)
L3 = tf.nn.tanh(tf.matmul(L2_drop, W3) + b3)
L3_drop = tf.nn.dropout(L3, keep_prob)

W4 = tf.Variable(tf.truncated_normal([1000, 10], stddev=0.1))
b4 = tf.Variable(tf.zeros([10]) +0.1)

# 使用softmax回归模型,y代表输出
y = tf.nn.softmax(tf.matmul(L3_drop, W4) + b4)

# 计算交叉墒
loss = tf.nn.softmax_cross_entropy_with_logits(labels=y_,logits=y)

# 使用梯度下降算法
global_step = tf.Variable(0) 

learning_rate = tf.train.exponential_decay(0.01, global_step, 100, 0.95, staircase=True) 
train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss,global_step=global_step)

# 初始化之前创建的变量的操作
init = tf.global_variables_initializer()

#启动初始化
with tf.Session() as sess:
    sess.run(init)

    # 开始训练模型,循环1000次
    for epoch in range(20):
        for i in range(1000):
            global_step = i
            batch_xs, batch_ys = mnist.train.next_batch(100)
            sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

        # 评估模型
        correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))

        # 计算正确预测项的比例
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

        test_acc = sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})
        train_acc = sess.run(accuracy, feed_dict={x: mnist.train.images, y_: mnist.train.labels})

        # 在session中启动accuracy,输入是MNIST中的测试集
        print("测试集准确率:" + str(test_acc) + "; 训练集准确率:" + str(train_acc))

打印,结果如下所示。

测试集准确率:0.8718; 训练集准确率:0.868145
测试集准确率:0.8959; 训练集准确率:0.895164
测试集准确率:0.9054; 训练集准确率:0.902255
测试集准确率:0.9111; 训练集准确率:0.909636
测试集准确率:0.9136; 训练集准确率:0.912764
测试集准确率:0.9197; 训练集准确率:0.914818
测试集准确率:0.9179; 训练集准确率:0.915491
测试集准确率:0.9173; 训练集准确率:0.917436
测试集准确率:0.9177; 训练集准确率:0.916309
测试集准确率:0.9229; 训练集准确率:0.915891

文章目录