ME 595R

HW 7: Deep Koopman

due 2/27/2025 before midnight via Learning Suite 25 possible points

We’ll use deep learning to learn a Koopman operator using the methodology of this paper (though we’ll simplify it somewhat for this homework). At a minimum you’ll want to refer to figure 1, the subsection “Deep learning to identify Koopman eigenfunctions” under Results, and the subsection “Explicit loss function” under Methods. We’ll make the following simplifications: we won’t need the auxiliary network to learn eigenvalues, and so won’t need to construct a block diagonal K (this is a nice approach for better accuracy and for explainability of the results, but we won’t worry about it in this case and will just learn the whole matrix K directly). I did not add the infinity norm loss in (15), for eq (13) I used the entire time horizon (so eq (13) and (14) use the same sum across time).

The data comes from glycolysis pathway dynamics. I ran the simulations and pretabulated the data here to save time. The following commands will load the data:

ntraj = 2148  # number of trajectories
nt = 50  # number of time steps
ny = 7  # number of states

tvec = np.linspace(0, 350, nt)
Y = np.loadtxt('kdata.txt').reshape(ntraj, nt, ny)
Ytrain = Y[:2048, :, :]  # 2048 training trajectories
Ytest = Y[2048:, :, :]  # 100 testing trajectoreis

You should be able to find a linear mapping (matrix K) that reasonably reproduces the dynamics in the testing data set. Plot the dynamics for the first trajectory in the dataset against your trained model (data with dashed line, and model solid lines). Only plot the first three states just to keep the plot less busy.

Tips:

• It’s generally a good idea to start by overfitting the data. Use only a relatively small number of training trajectories so that things run fast, and make sure you can train the model to reproduce your training data (ignore testing data for now). Until you can fit the training data there is no point trying to generalize to the testing data. And by keeping it small you can iterate quickly and make sure your loss functions, etc. are setup properly.
• I’d then add more training data and modify hyperparameters until you can get good predictions (still with training data). Then try to generalize to the testing data. You might not need to use all 2048 data trajectories. I just keep doubling the number of training data points until I was able to get my test set error down.
• A GPU (google colab) will be helpful when you start using more data.
• For the first set of epochs I only train the autoencoder, and then add on the losses for linearity and prediction.
• If you create a nn.Parameter for K within your nn.Module then when you pass model.parameters() to your optimizer it automatically includes K along with all the model weights and biases. Or, even easier, you can create a linear layer with bias=False.
• When you initialize K, keep its weights small. Since \(x_{k+1} = K x_k\), if the entires in \(K\) are large your dynamics will blow up quickly.
• If you’re struggling with generalization (training error is going down but testing error is going up) a few techniques you can try including using more data (if you haven’t already maxed that out), using dropout, adding a penalty on model weights.