This article will first introduce gradient descent, and then go through most of popular optimization methods, such as:

• SGD
• RMSprop
• Adam

Notes about using Tensorflow

Quick notes

trainable: If True, the default, also adds the variable to the graph collection GraphKeys.TRAINABLE_VARIABLES. This collection is used as the default list of variables to use by the Optimizer classes.

Tensorflow gpu auto growth

config = tf.ConfigProto()
config.gpu_options.allow_growth = True

Reset default graph

tf.reset_default_graph()

Tensorboard

with tf.Session(config=config) as sess:

    # ...

    writer = tf.summary.FileWriter('./graphs/test', sess.graph)
    writer.flush()
    writer.close()

Startup tensorboard:

Notes about Keras.

• Keras version: 2.0.8.
  • Keras 2.0.8 Documentation

Problems encountered

2017-12-04 · Get variable’s value in middle layer

Problem description: I want to get the value of a variable in the model.

Reference: How can I get hidden layer representation of the given data? #41

The simplest way is using the same code of the original model, and

# replace the output with the variable you want
# code ...

new_model.set_weights(trained_model.get_weights())
new_model.predict(input_data, batch_size=32)

Note the batch_size is import for large amount of samples. The K.function() mentioned in the issue #41 raised OOM exception. Of course you can split data into batches by yourself and use the K.function() method, but the method showed above is more convinient for me in my case.

2018年4月22日，第二次来首都博物馆观看北京四合院的展览。

第一次来是3月17号，来首博看完西藏特展——天路文化后，转悠至此，被一位东城大叔饱含故事的讲解深深吸引。 回去之后，又找来贾珺的北京四合院读了个痛快，不禁被古建筑里的文化和艺术深深吸引。此次前来，重温展览的同时，为了留下一份这些资料的记录。

2018年5月5日，从青年公寓出发步行至故宫，这次前往故宫的主要目的是单士元先生的纪念展。

展览以“紫禁城营造与保护”为主题，展厅一层以存世样式雷图样和故宫藏烫样为展览重点，展厅二层则以古建守护者为题，系统讲述故宫人的成就与品格。 上下楼梯左侧是屋角仙人和走兽的实物展示，右侧是一些建筑构件的实物展示。

This article will introduce:

• What is boosting.
• Common boosting algorithms, including:
  • AdaBoost
  • Gradient Boosting

Notes for Golang.

Links

• Go lang cheat sheet [Github]

Notes for Julia.

Links

• Docs
• Introduction to Julia for Pythonistas [YouTube] [Notebook]