Keras

Notes about Keras.

Problems encountered

2017-12-04 · Get variable’s value in middle layer

Problem description: I want to get the value of a variable in the model.

Reference: How can I get hidden layer representation of the given data? #41

The simplest way is using the same code of the original model, and

# replace the output with the variable you want
# code ...

new_model.set_weights(trained_model.get_weights())
new_model.predict(input_data, batch_size=32)

Note the batch_size is import for large amount of samples. The K.function() mentioned in the issue #41 raised OOM exception. Of course you can split data into batches by yourself and use the K.function() method, but the method showed above is more convinient for me in my case.

Example case: I want to get the output value of pooling layer.

  • get_model() function return a model for training
  • train_model()
  • get_p_out() function almost have the same code with get_model() except
    • the input parameters
    • the Model()’s output parameter
    • set_weights() from trained model
    • use the new model to predict value

Code in gist

2018-02-27 · Training using large dataset

Problem description: Sometimes we need to use large dataset to train our model, but large dataset can’t fit into memory.

Reference: For large datasets, which to use: fit or train_on_batch?

def generate_arrays_from_file(path):
    while 1:
        f = open(path)
        for line in f:
            # create numpy arrays of input data
            # and labels, from each line in the file
            x, y = process_line(line)
            img = load_images(x)
            yield (img, y)
        f.close()

model.fit_generator(generate_arrays_from_file('/my_file.txt'),
        samples_per_epoch=10000, nb_epoch=10)

fit_generator: The generator is run in parallel to the model, for efficiency. For instance, this allows you to do real-time data augmentation on images on CPU in parallel to training your model on GPU.

Quick notes

Metric

As of Keras 2.0, precision and recall were removed from the master branch. We need to implement them.

They are removed because they should be used with all results (globally) rather than one mini-batch:

Basically these are all global metrics that were approximated batch-wise, which is more misleading than helpful. This was mentioned in the docs but it’s much cleaner to remove them altogether. It was a mistake to merge them in the first place.

@fachollet issue #5794

Old code can be found in: issue #5400 or Removed batchwise metrics

Code in my gist

Keras Syntax

The syntax shown below is very common used in Keras. We can use it in this way the usage of Python’s functional programming.

from keras.layers import Input, Dense

inputs = Input(shape=(784,))
x = Dense(64, activation='relu')(inputs)

Take keras.layers.merge.Concatenate as example, the code behind may be like this.

def Concatenate(axis=-1):
    def afunc(x):
        return concatenate(x, axis=-1)
    return afunc

So Concatenate(axis=-1)([a, b]) is equal with concatenate([a, b], axis=-1).

Just guess, haven’t checked the source code.

Tensorboard

from datetime import datetime
from keras.callbacks import TensorBoard

def get_log_path(key):
    name = datetime.now().strftime("%Y-%m-%d-%H-%M-%S")
    return './graphs/' + key + '-' + name

tensorboard = TensorBoard(log_dir=get_log_path('res_net'))
model.fit(..., callbacks=[tensorboard])

Deep learning models in Keras

Trained image classification models for Keras