You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for your great implementation!
I' ve tried to solve classification problem whose input data have the shape of 1000*221 by DEC model.
I want to train my data with over 80 thousand data (very large size [8000000,1000,221],dtype=float32 about 60GB ), so it's not possible load whole dataset into python array.
After googling, I found tf.TFRecord helps me to get out this capacity problem.
I followed the tutorial in the official TensorFlow site to write TFRecord file and I can load the TFReocrd into the conventional Keras Model. However, I can't find how to feed into the DEC-model. The input (mnist) of DEC-model is one numpy file that has the shape [70000,784].
Thanks for your great implementation!
I' ve tried to solve classification problem whose input data have the shape of 1000*221 by DEC model.
I want to train my data with over 80 thousand data (very large size [8000000,1000,221],dtype=float32 about 60GB ), so it's not possible load whole dataset into python array.
After googling, I found tf.TFRecord helps me to get out this capacity problem.
I followed the tutorial in the official TensorFlow site to write TFRecord file and I can load the TFReocrd into the conventional Keras Model. However, I can't find how to feed into the DEC-model. The input (mnist) of DEC-model is one numpy file that has the shape [70000,784].
Like flowing:
dataset = tf.data.TFRecordDataset(filenames=[filenames])
parsed_dataset = dataset.map(_parse_function, num_parallel_calls=8)
final_dataset = parsed_dataset.shuffle(buffer_size=number_of_sample).batch(10)
iterator = dataset.make_one_shot_iterator()
parsed_record = iterator.get_next()
feature, label = parsed_record['feature'], parsed_record['label']
#keras
inputs = keras.Input(shape=(1000,221 ), name='feature', tensor=feature)
model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
loss='categorical_crossentropy',
metrics=['accuracy','categorical_crossentropy'],
target_tensors=[label]
)
train_model.fit(epochs= 30,
steps_per_epoch= 800000/256)
The text was updated successfully, but these errors were encountered: