# Smoke test passes but fails normal

**URL:** <https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966>\
**Category:** Youth Mental Health: Automated Abstraction\
**Created:** [November 15, 2024, 8:58am UTC](https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966 "2024-11-15T08:58:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ethan-Sinson](https://avatars.discourse-cdn.com/v4/letter/e/f0a364/32.png) [@Ethan-Sinson](https://community.drivendata.org/u/Ethan-Sinson)\
**Post date:** [November 15, 2024, 8:58am UTC](https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966/1 "2024-11-15T08:58:43Z")

</div>

I noticed that my smoke test passes but whenever I submit I get “IDs for submission are not correct.”

What could possibly lead to this issue if both results follow the submission format?

Minor addition is the `test_features.csv` given by the competition?

---

<div class="post-metadata">

**Author:** ![kwetstone](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@kwetstone](https://community.drivendata.org/u/kwetstone)\
**Post date:** [November 15, 2024, 2:23pm UTC](https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966/2 "2024-11-15T14:23:56Z")

</div>

@Ethan-Sinson This means that the IDs (`uid` column) do not match the `submission_format.csv` provided in the competition runtime for the full test set. There are a number of reasons this could be happening (eg. correctly ordered rows, missing some uids, etc).

I recommend checking out the [benchmark blog post](https://drivendata.co/blog/automated-abstraction-benchmark) for some handy code to make sure your rows and columns are in the right order. E.g.:

```auto
# Make sure the row order is the same as in the submission format
parsed_preds = parsed_preds.loc[features.index]

```

---

<div class="post-metadata">

**Author:** ![Ethan-Sinson](https://avatars.discourse-cdn.com/v4/letter/e/f0a364/32.png) [@Ethan-Sinson](https://community.drivendata.org/u/Ethan-Sinson)\
**Post date:** [November 15, 2024, 5:16pm UTC](https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966/3 "2024-11-15T17:16:15Z")

</div>

@kwetstone I’ll check this out, but I also do want to know whether test\_features.csv should be on the data download page?

---

<div class="post-metadata">

**Author:** ![kwetstone](https://avatars.discourse-cdn.com/v4/letter/k/ee7513/32.png) [@kwetstone](https://community.drivendata.org/u/kwetstone)\
**Post date:** [November 15, 2024, 5:33pm UTC](https://community.drivendata.org/t/smoke-test-passes-but-fails-normal/10966/4 "2024-11-15T17:33:56Z")

</div>

@Ethan-Sinson This is a [code execution challenge](https://www.drivendata.org/competitions/295/cdc-automated-abstraction/page/923/), so test features are kept confidential. Your code will be used to generate predictions on the test features in the code execution runtime. The submission format is shared, so you can see the expected `uid`s in the full test set.