Track B: clarification over target label

PETs Prize Challenge
1

Hi,
Apologies if this has been asked previously, but could you clarify what the target label is for the pandemic track. On the data overview page it is described in two places as:

  • Your task is to predict a risk score for the binary disease state (infected or not infected) of each individual in the final week of the simulation

  • The target variable for the modeling task is a risk score (between 0.0 and 1.0) for each individual in the population. That risk score corresponds to a confidence that that individual enters into an symptomatic infected I disease state at any time during the final week of the simulation (between days 56 and 63).

Specifically, if someone has a status of “I” on both days 56 and 57, what label would they have in the target? The first statement above would suggest 1 whilst the second would suggest 0.
I would assume it is the second case. However if that is the case I believe there to be an error in the va_disease_outcome_target.csv data since there are a few individuals who have positive labels but have either “R” or “I” status by day 56

Thank you

2

Hi @domdathan,

Here is a response to a related question that we had provided to another team:

We have confirmed with the data provider that the target represents a transition to the “I” or infected state in the final/target week. Therefore, anyone who is in state “I” before the final week, and continues to be in state “I” in the target week, should not be labeled as “Infected” in “va_disease_ outcome_target.csv”. It does appear that there is one mistake; the person with pid = 5614899 transitioned to state “I” on day 53, and therefore should have an indicator value of “0” in the ground truth table, but is instead labeled “1”.

If it is the case also that there is a small number of individuals who are status R and have been labeled as 1 in the test ground truth, you should consider it to be an error in the dataset and label noise. Per this thread, state transitions after R are not allowed.

1 Like
3

Makes sense, thanks for clarifying @jayqi!