1

In ch17 of the textbook 'hands on machine learning' by Aurélien Geron, the results of the functional API of the autoencoder code (which I implemented myself) are different from the results of Sequential (example code in the textbook).

The code below is an example code from Textbook ch17.

# https://github.com/ageron/handson-ml3/blob/main/17_autoencoders_gans_and_diffusion_models.ipynb
import tensorflow as tf

tf.random.set_seed(42)  # extra code – ensures reproducibility on CPU

encoder = tf.keras.Sequential([tf.keras.layers.Dense(2)])
decoder = tf.keras.Sequential([tf.keras.layers.Dense(3)])
autoencoder = tf.keras.Sequential([encoder, decoder])

optimizer = tf.keras.optimizers.SGD(learning_rate=0.5)
autoencoder.compile(loss="mse", optimizer=optimizer)

import numpy as np
from scipy.spatial.transform import Rotation

m = 60
X = np.zeros((m, 3))  # initialize 3D dataset
np.random.seed(42)
angles = (np.random.rand(m) ** 3 + 0.5) * 2 * np.pi  # uneven distribution
X[:, 0], X[:, 1] = np.cos(angles), np.sin(angles) * 0.5  # oval
X += 0.28 * np.random.randn(m, 3)  # add more noise
X = Rotation.from_rotvec([np.pi / 29, -np.pi / 20, np.pi / 4]).apply(X)
X_train = X + [0.2, 0, 0.2]  # shift a bit

history = autoencoder.fit(X_train, X_train, epochs=500, verbose=False)
codings = encoder.predict(X_train)

fig = plt.figure(figsize=(4,3))
plt.plot(codings[:,0], codings[:, 1], "b.")
plt.xlabel("$z_1$", fontsize=18)
plt.ylabel("$z_2$", fontsize=18, rotation=0)
plt.grid(True)
save_fig("linear_autoencoder_pca_plot")
plt.show()

[![linear_autoencoder_pca_plot](https://i.sstatic.net/bN2VI.png)](https://i.sstatic.net/bN2VI.png)

The code below is my conversion of this example code into a functional API.

from tensorflow.keras.layers import Dense
from tensorflow.keras import Input, Model
input1 = Input(shape=3)
x = Dense(2, activation='linear', name="intermediate")(input1)
x = Dense(3, activation='linear')(x)
model_autoencoder_funcAPI = Model(inputs=input1, outputs=x)
optimizer1 = tf.keras.optimizers.SGD(learning_rate=0.5)
model_autoencoder_funcAPI.compile(loss='mse', optimizer=optimizer1)
history_funcAPI = model_autoencoder_funcAPI.fit(X_train, X_train, epochs=501)
layer_name = 'intermediate'
intermediate_layer_model = Model(inputs=input1, outputs=model_autoencoder_funcAPI.get_layer(layer_name).output)
codings_autoencoder_funcAPI = intermediate_layer_model.predict(X_train)

fig = plt.figure(figsize=(4,3))
plt.plot(codings_autoencoder_funcAPI[:,0], codings_autoencoder_funcAPI[:, 1], "b.")
plt.xlabel("$z_1$", fontsize=18)
plt.ylabel("$z_2$", fontsize=18, rotation=0)
plt.grid(True)
save_fig("ch17_linear_autoencoder_pca_plot_funcAPI")
plt.show()
[![ch17_linear_autoencoder_pca_plot_funcAPI](https://i.sstatic.net/6uuat.png)](https://i.sstatic.net/6uuat.png)

The results of the two codes are different. Where is the problem?

Thank you.

When implemented with the functional API code, I expected the same results as the results of the Sequential code.

================================================================ Solved! Following user3128's advice, I added tf.random.set_seed(42) before building the model of the second code, and the result was the same as that of the first code.

3
  • Try setting the random seed again before you build the second model. I think that would ensure the random initialisation of weights for the second model will match that of the first model. Commented Dec 4, 2023 at 11:04
  • 1
    @user3128 Thank you bro. I added tf.random.set_seed(42) to the beginning of the second code and got the same result! Thanks to your comment, I learned more about TF. Commented Dec 8, 2023 at 6:36
  • My pleasure @IKAHN, glad I could assist. Commented Dec 8, 2023 at 8:52

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.