# Did anyone notice that Benchmark LSTM solution is not correct?

**URL:** <https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808>\
**Category:** Cold Start Energy Forecasting\
**Created:** [October 17, 2018, 2:11pm UTC](https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808 "2018-10-17T14:11:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![snanurag](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@snanurag](https://community.drivendata.org/u/snanurag)\
**Post date:** [October 17, 2018, 2:11pm UTC](https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808/1 "2018-10-17T14:11:21Z")

</div>

In LSTM benchmark solution, when they are training the model, the input consumption data order is (latest … last) but in the prediction, they are using input consumption order (last… latest). It seems to need a flip of array.

---

<div class="post-metadata">

**Author:** ![unnir](https://avatars.discourse-cdn.com/v4/letter/u/aca169/32.png) [@unnir](https://community.drivendata.org/u/unnir)\
**Post date:** [October 22, 2018, 10:40am UTC](https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808/2 "2018-10-22T10:40:52Z")

</div>

Could you please share the code? I just want to understand what do you mean by the a flip of array?

---

<div class="post-metadata">

**Author:** ![snanurag](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@snanurag](https://community.drivendata.org/u/snanurag)\
**Post date:** [October 22, 2018, 3:32pm UTC](https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808/3 "2018-10-22T15:32:39Z")

</div>

In their benchmark solution, check generate\_hourly\_forecast function and see what I did with the X array below.

```auto
def generate_hourly_forecast(num_pred_hours, consumption, model, scaler, lag):
    """ Uses last hour's prediction to generate next for num_pred_hours, 
        initialized by most recent cold start prediction. Inverts scale of 
        predictions before return.
    """
    # allocate prediction frame
    preds_scaled = np.zeros(num_pred_hours)
    
    # initial X is last lag values from the cold start
    X = scaler.transform(consumption.values.reshape(-1, 1))[-lag:]
    
    # forecast
    for i in range(num_pred_hours):
        # predict scaled value for next time step
        X1 = np.flip(X.ravel(),axis=0)
        yhat = model.predict(X1.reshape(1, 1, lag), batch_size=1)[0][0]
        preds_scaled[i] = yhat
        
        # update X to be latest data plus prediction
        X = pd.Series(X.ravel()).shift(-1).fillna(yhat).values

    # revert scale back to original range
    hourly_preds = scaler.inverse_transform(preds_scaled.reshape(-1, 1)).ravel()
    return hourly_preds

```

---

<div class="post-metadata">

**Author:** ![kyz682](https://avatars.discourse-cdn.com/v4/letter/k/4491bb/32.png) [@kyz682](https://community.drivendata.org/u/kyz682)\
**Post date:** [October 23, 2018, 2:02am UTC](https://community.drivendata.org/t/did-anyone-notice-that-benchmark-lstm-solution-is-not-correct/2808/4 "2018-10-23T02:02:22Z")

</div>

Good catch! I didn’t notice that when I first ran through the benchmark code, but it looks like the input array is not in proper order of (t-1, t-2, t-3, t-4…). You would have to shift it by 1 instead of -1 though in order to update the latest prediction to t-1.