# Forecast Stage Leaderboard (Unofficial)

**URL:** <https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736>\
**Category:** Water Supply Forecast Rodeo\
**Created:** [July 4, 2024, 1:25pm UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736 "2024-07-04T13:25:14Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rasyidstat](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@rasyidstat](https://community.drivendata.org/u/rasyidstat)\
**Post date:** [July 4, 2024, 1:25pm UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/1 "2024-07-04T13:25:15Z")

</div>

Hi,

There’s no live LB for the Forecast Stage since the full ground truth will be released next month. However, we can peek and guess the LB based on the latest condition up until now with quite good confidence (expected 5% error). Here is the “unofficial” LB based on MAE only (volume\_50) using the data from this [page](https://www.drivendata.org/competitions/259/reclamation-water-supply-forecast/page/876/).

![image](https://canada1.discourse-cdn.com/flex028/uploads/drivendata1/original/2X/b/b5353748568b8a15bf7b1f3d95bc54a3a0d17c54.png)

Take this with a grain of salt as it’s only based on MAE and the latest condition until June 2024. I expect that the top ranking will be around 60-ish and it’s still possible to have a 50-ish score for this water year season.

---

<div class="post-metadata">

**Author:** ![rasyidstat](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@rasyidstat](https://community.drivendata.org/u/rasyidstat)\
**Post date:** [July 21, 2024, 1:06pm UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/2 "2024-07-21T13:06:08Z")

</div>

_Updated at 2024-07-25_  
Here is the update with the latest condition and forecast in July. All issue dates are available to be scored  
 ![image](https://canada1.discourse-cdn.com/flex028/uploads/drivendata1/original/2X/6/64077fc625f3a5e27df30a126c31b8d8a36f22fe.png)

---

<div class="post-metadata">

**Author:** ![RomanChernenko](https://avatars.discourse-cdn.com/v4/letter/r/76d3ee/32.png) [@RomanChernenko](https://community.drivendata.org/u/RomanChernenko)\
**Post date:** [July 22, 2024, 7:08am UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/3 "2024-07-22T07:08:02Z")

</div>

Hello @rasyidstat  
Thank you for sharing this! I tried to calculate a similar leaderboard table based on NRCS benchmark data and got slightly different results. But seems the benchmark data is too outdated and not so suitable to use as approximated GT. What values do you use as GT for comparison here?

---

<div class="post-metadata">

**Author:** ![rasyidstat](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@rasyidstat](https://community.drivendata.org/u/rasyidstat)\
**Post date:** [July 22, 2024, 10:27am UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/4 "2024-07-22T10:27:39Z")

</div>

Hi @RomanChernenko  
I mainly used the RFC benchmark since the forecast has been updated up until today. For example: [https://www.nwrfc.noaa.gov/water\_supply/ws\_forecasts.php?id=HHWM8](https://www.nwrfc.noaa.gov/water_supply/ws_forecasts.php?id=HHWM8)  
For some locations, I used various assumptions for the ground truth such as my latest forecast, latest median forecast of all participants, etc.

Again, take this with a grain of salt. There are only 26 ground truths and some adjustments/revisions of natural water supply volume in locations with high volume can change the result significantly.

---

<div class="post-metadata">

**Author:** ![RomanChernenko](https://avatars.discourse-cdn.com/v4/letter/r/76d3ee/32.png) [@RomanChernenko](https://community.drivendata.org/u/RomanChernenko)\
**Post date:** [July 25, 2024, 3:49pm UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/5 "2024-07-25T15:49:32Z")

</div>

Hi @rasyidstat  
Could you please recalculate the results again? The forecast prediction page already has the data for 24th of July.

---

<div class="post-metadata">

**Author:** ![rasyidstat](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@rasyidstat](https://community.drivendata.org/u/rasyidstat)\
**Post date:** [July 25, 2024, 10:37pm UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/6 "2024-07-25T22:37:51Z")

</div>

Hi @RomanChernenko, I already updated it above

> [@Forecast Stage Leaderboard (Unofficial)](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/2):
>
> Updated at 2024-07-25 Here is the update with the latest condition and forecast in July. All issue dates are available to be scored [image]

---

<div class="post-metadata">

**Author:** ![RomanChernenko](https://avatars.discourse-cdn.com/v4/letter/r/76d3ee/32.png) [@RomanChernenko](https://community.drivendata.org/u/RomanChernenko)\
**Post date:** [July 26, 2024, 8:28am UTC](https://community.drivendata.org/t/forecast-stage-leaderboard-unofficial/10736/7 "2024-07-26T08:28:27Z")

</div>

Very interesting to see how top3 average model overperforms all other models.
