# ReUse of train-data generated features for test phase

**URL:** <https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495>\
**Category:** PETs Prize Challenge\
**Created:** [January 23, 2023, 3:55pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495 "2023-01-23T15:55:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kingaj12](https://avatars.discourse-cdn.com/v4/letter/k/a87d85/32.png) [@kingaj12](https://community.drivendata.org/u/kingaj12)\
**Post date:** [January 23, 2023, 3:55pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495/1 "2023-01-23T15:55:56Z")

</div>

In the phase-one notebook, the features generated on training data (`sender\_currency\_freq’ etc) were used both to fit the RF or XGBoost models _and_ for the prediction on the test data.

For phase-two: is this reuse of train-data generated features also expected by the evaluators?

It makes sense, since this is a time-series application, but it also seems that there is no facility in the code template for the transmission of train-data generated features.

---

<div class="post-metadata">

**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\
**Post date:** [January 23, 2023, 4:31pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495/2 "2023-01-23T16:31:28Z")

</div>

Hi @kingaj12,

I don’t see what you’re saying reflected in the notebook. The train features and test features are loaded as separate dataframes `train` and `test` cell 4. While it’s true that feature engineering is often applied to both these dataframes in the same cell in the notebook, they are never combined into a single dataframe and manipulated together.

If you’re referring to features like `sender_currency_freq` which have parameters that are fitted to the train data and then applied to the test data, that is fairly normal in machine learning problems. That isn’t really reuse of training features—the features used for prediction on the test split are still test features (`X_test`, which comes from the `test` dataframe).

Regardless, if you need to store any information between the train stage and test stage, you can use the `client_dir` and `server_dir` directories that are provided to the client and strategy factory functions, respectively. This is, in fact, the expected way for you to save a trained model so that you can use that model later in test. This is documented [here](https://www.drivendata.org/competitions/105/nist-federated-learning-2-financial-crime-federated/page/587/#test).

In terms of how to implement the saving and reloading of feature engineering parameters for use during the test stage, that’s up to you. I’m personally a fan of [scikit-learn’s Pipeline API](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html) for bundling feature engineering together with the model for easy saving, and that is in fact what is done in the example ([here](https://github.com/drivendataorg/pets-prize-challenge-runtime/blob/d140e87d0bdc48607ac12809e5b7bbbdb9737859/examples_src/fincrime/model.py#L11-L16)). You should do whatever works for you.

Hope that helps. Let me know if there’s anything else that we can help clarify.

---

<div class="post-metadata">

**Author:** ![kingaj12](https://avatars.discourse-cdn.com/v4/letter/k/a87d85/32.png) [@kingaj12](https://community.drivendata.org/u/kingaj12)\
**Post date:** [January 23, 2023, 5:40pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495/3 "2023-01-23T17:40:09Z")

</div>

Hi @jayqi

The following code fragment computes `sender_currency_freq` inside the loop using `train["sender_currency"]` and then maps this `sender_currency_freq` to both the `train` and `test` dataframes:

```
# Sender-Currency Frequency and Average Amount per Sender-Currency
train["sender_currency"] = train["Sender"] + train["InstructedCurrency"]
test["sender_currency"] = test["Sender"] + test["InstructedCurrency"]

sender_currency_freq = {}
sender_currency_avg = {}

for sc in set(list(train["sender_currency"].unique()) + list(test["sender_currency"].unique())):
    sender_currency_freq[sc] = len(train[train["sender_currency"] == sc])
    sender_currency_avg[sc] = train[train["sender_currency"] == sc]["InstructedAmount"].mean()

train["sender_currency_freq"] = train["sender_currency"].map(sender_currency_freq)
test["sender_currency_freq"] = test["sender_currency"].map(sender_currency_freq)

train["sender_currency_amount_average"] = train["sender_currency"].map(sender_currency_avg)
test["sender_currency_amount_average"] = test["sender_currency"].map(sender_currency_avg)

```

From your response, I gather this is not what was intended!

---

<div class="post-metadata">

**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\
**Post date:** [January 23, 2023, 6:02pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495/4 "2023-01-23T18:02:29Z")

</div>

Hi @kingaj12,

The way it’s implemented **is** intended, and is a normal way of implementing fitted feature engineering in machine learning.

You can consider the `sender_currency_freq` and `sender_currency_avg` mappings to be fitted on the training data—the values for each sender currency are effectively like model parameters.

It would be bad practice in this situation to calculate new frequencies and averages based on the test data—this does not treat test observations independently. It means you would get different answers if you run inference on the whole test set at once, vs. feeding the test observations through the model pipeline one-at-a-time. This is also a form of data leakage—you’ve fit what is basically part of your model on test data that you are evaluating on.

---

<div class="post-metadata">

**Author:** ![jayqi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/jayqi/32/924_2.png) [@jayqi](https://community.drivendata.org/u/jayqi)\
**Post date:** [January 23, 2023, 6:21pm UTC](https://community.drivendata.org/t/reuse-of-train-data-generated-features-for-test-phase/8495/5 "2023-01-23T18:21:23Z")

</div>

For reference, here is some discussion about this type of thing:

- [machine learning - Feature Engineering for missing values over test and train data together or separately - Cross Validated](https://stats.stackexchange.com/a/503678)
- [cross validation - Imputation before or after splitting into train and test? - Cross Validated](https://stats.stackexchange.com/a/95088)