All Questions
Tagged with tf.keras recurrent-neural-network
30
questions
6
votes
1
answer
678
views
Cascade multiple RNN models for N-dimensional output
I'm having some difficulty with chaining together two models in an unusual way.
I am trying to replicate the following flowchart:
For clarity, at each timestep of Model[0] I am attempting to generate ...
- 16.9k
4
votes
2
answers
2k
views
Saving a Tensorflow Keras model (Encoder - Decoder) to SavedModel format
I've written (with help from TF tutorials) an image captioning model which uses an encoder-decoder model with attention.
Now, I want to convert it to TfLite and eventually deploy it in Flutter.
I'm ...
- 41
4
votes
1
answer
195
views
Can I change the statefulness of RNN after training?
If I build and train my RNN based model with stateful=False, can I simply do (e.g.):
model.layers[0].stateful = True
And have it take effect as might be expected for use in prediction? I ask ...
- 2,170
4
votes
2
answers
1k
views
What does Keras do with the initial values of cell & hidden states (RNN, LSTM) for inference?
Assuming training is finished: what values does Keras use for the 0th cell state and hidden states at inference (in LSTM and RNN layers)? I could think of at least three scenarios, and could not find ...
- 487
4
votes
1
answer
2k
views
Elastic Weight Consolidation Algorithm Implementation in Keras
I am working on an LSTM based model to predict logs-anomaly. My model architecture is as given:
______________________Layer (type) Output Shape Param # ================================================...
- 807
3
votes
2
answers
2k
views
LSTM - Use deltaTime as a feature? How to handle irregular timestamps?
I'm trying to create a LSTM for classification of data sequences.
The data structure of every training input that I would use is:
[[
[deltaX,deltaY,deltaTime],
[deltaX,deltaY,deltaTime],...
],class]
...
- 430
2
votes
1
answer
862
views
What is the connections between two stacked LSTM layers?
The question is like this one What's the input of each LSTM layer in a stacked LSTM network?, but more into implementing details.
For simplicity how about 4 units and 2 units structures like the ...
- 468
2
votes
1
answer
2k
views
Converting Keras RNN model to TensorFlow Lite model in TF2
I am currently trying to convert a RNN model to TF lite. After multiple failed attempts I tried running the example given in the repository found here. This threw errors too due to changes in the ...
- 495
2
votes
0
answers
319
views
How to freeze tensorflow variables inside tf.keras framework on eager execution mode?
I'm trying to fine tune the input weights in a recurrent cell without letting the backpropagation affect previous states (kind of truncated backpropagation with n = 1). I'm using tf.keras and eager ...
- 21
1
vote
1
answer
982
views
Tensorflow.keras: RNN to classify Mnist
I am trying to understand the tensorflow.keras.layers.SimpleRNN by building a simple digits classifier. The digits of Mnist dataset are of size 28X28. So the main idea is to present each line of the ...
- 3,229
1
vote
1
answer
241
views
Meaning of 2D input in Keras LSTM
In Keras, LSTM is in the shape of [batch, timesteps, feature]. What if I indicate the input as keras.Input(shape=(20, 1)) and feed a matrix of (100, 20, 1) as input? What's the number of batch that it'...
- 53
1
vote
1
answer
608
views
Can I split my long sequences into 3 smaller ones and use a stateful LSTM for 3 samples?
I am doing a time-series sequence classification problem.
I have 80 time-series all length 1002. Each seq corresponds to 1 of 4 categories (copper, cadmium, lead, mercury). I want to use Keras LSTMs ...
- 1,087
1
vote
0
answers
167
views
Output layer of LSTM (many to many)
I want to use LSTM to predict Y (size = [batch_size, time_length=1, feature_dim=3]) from X (size = [batch_size, time_length=42, feature_dim=12]). The problem is using data (12 features) during past 42 ...
- 197
1
vote
1
answer
416
views
Fitting a custom (non-sequential) stateful RNN (GRU) model
I am facing some problems in training the following GRU model, which has to be stateful and output the hidden state.
import numpy as np
import tensorflow as tf #2.1.0
from tensorflow import keras
...
- 132
1
vote
0
answers
284
views
Replacing `dynamic_rnn` with `tf.keras.layers.RNN` giving different results
For a simple many to one time-series prediction problem where I want to predict a value given previous 4 values, I am using a code which employs dynamic_rnn. I want to replace this with tf.keras....
- 416
1
vote
0
answers
357
views
Implementing Sequential Variational Autoencoder (State-Space Model) on TensorFlow
I'm currently trying to implement a version of variational autoencoder in a sequential setting. I work on TensorFlow with eager execution mode.
As the problem setting, I have two sequences of ...
1
vote
2
answers
849
views
Tensorflow 2 custom LSTM Cell initial states
I try to implement a custom LSTM cell. Firstly, I try to reproduce the original LSTM cell before I put customization. However, I run into a problem that the initial state is a single tensor instead of ...
- 11
0
votes
1
answer
723
views
In a Keras custom RNN cell, what are the dimensions of the inputs and outputs?
The custom cell takes (input,state) and generates (output,state). I believe input is a tensor, and state is a list of tensors. From fighting through error messages it appears that the tensors carry ...
- 2,170
0
votes
1
answer
298
views
How to use Tokenizer (Keras)? Unable to generate tokens on Character level
goal: vectorizing on character-level
problem: output is not a unique number per character/letter, instead all letters are converted to 1
Question: What is wrong with my code?
I have a dataframe (df). ...
- 55
0
votes
1
answer
127
views
Differentiation Issue in Predictive Alignment for Attention Implementation
I am trying to implement local-p attention based on this paper: https://arxiv.org/pdf/1508.04025.pdf Specifically, equation (9) derives a alignment position based on taking the sigmoid of some non-...
- 254
0
votes
0
answers
48
views
Tensorflow/Keras LSTM RNN model for consumption forecast not learning/underfitting
Hi everyone I need help!
I have build a LSTM RNN model with Tensorflow and Keras to predict the electricity consumption of a building. I have half hourly data since august 2021 (just above 4 million ...
- 1
0
votes
0
answers
16
views
Call tf.stop_gradient() on hidden states of recurrent neural network in Tensorflow/Keras
I have a question on Tensorflow/Keras and the use of tf.stop_gradient() in connection to RNN:s. In this pytorch guide, they call .detach() on the hidden states that the model outputs before passing it ...
- 211
0
votes
0
answers
13
views
RNN where features are sequences with different lengths
I have a sequential RNN using Keras. Some of my features are sequences with different lengths (eg time series where data is collected daily or monthly) and some other features are scalars. Would this ...
- 71
0
votes
0
answers
107
views
Error while importing tensorflow: No module named tensorflow.tsl
I've working on RNN_LSTM and the codes were working well just a month ago, but now gives errors, I tried to fix but no result....
Found out the error is on TensorFlow
I installed successfully but can'...
0
votes
0
answers
60
views
Understanding Keras Layer Shape
My features/targets look like:
x[0] = [10, 15, 13]
y[0] = [1, 4]
The numbers represent lookup indexes for words in english and french.
Here's the shape of my input and training data:
input: (137861, ...
- 402
0
votes
1
answer
926
views
Input shape for a RNN in keras
My data set has the following shapes:
y_train.shape,y_val.shape
((265, 2), (10, 2))
x_train.shape, x_val.shape
((265, 4), (10, 4))
I'm trying to use a simple RNN model
model=models.Sequential([...
- 1,298
0
votes
0
answers
37
views
Keras multi-input model with tf.keras.utils.Sequence [duplicate]
I am trying to train an RNN model in Keras to predict the next element in a sequence (e.g. given [a,b,c,d] predict e). Each element in the sequence has two components (lets call them x and y) of ...
- 840
0
votes
1
answer
270
views
Keras(TensorFlow backend) multi-gpu model(4gpus) is failing when using Masking on input of LSTM network
Masking the input layer in LSTM and trying to run on multi-GPU-model with fi_genrator of Keras using TensorFlow background is throwing errors.
created a fit_generator for an LSTM, and code runs on ...
- 51
-1
votes
1
answer
87
views
loss value deep learning model is inf
I train a RNN deep learning model as bellow:
model = Sequential()
initializer = tf.keras.initializers.RandomNormal(mean=.5, stddev=1)
model.add(LSTM(512, return_sequences=True, dropout=0.2,input_shape=...
- 1
-1
votes
1
answer
933
views
TypeError: 'tuple' object is not callable?
I am trying to plot the Model accuracy and Model loss learning curves after I have successfully trained my LSTM model to see the pattern of the learning curves (to see if its overfitting or ...
