Manually calculate RMSE

Instead of using model.evaluate I want to use model.predict. In this case I get a numpy array of 17_739, 2 predictions (t0 and t1) given my test set of 18_000.
What if I want to manually calculate the RMSE (using RootMeanSquaredError from keras or mean_squared_error from sklearn)?

test_ds = timeseries_dataset_from_df(test, data_config["batch_size"])
preds_1 = model.predict(test_ds)
(17739, 2)

Ok, solved. I got the results from test_ds.as_numpy_iterator()