# Failed without log

**URL:** <https://community.drivendata.org/t/failed-without-log/8296>\
**Category:** Video Similarity Challenge\
**Created:** [December 16, 2022, 1:54am UTC](https://community.drivendata.org/t/failed-without-log/8296 "2022-12-16T01:54:23Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 1:54am UTC](https://community.drivendata.org/t/failed-without-log/8296/1 "2022-12-16T01:54:23Z")

</div>

In my second submission to the descriptor track, it failed without logs. I only see:

```auto
unable to retrieve container logs for containerd://e90b0c5af870eb6bfa2c0e234831696d0dc870de905725df9ce67cbfd0af9e6f

```

Could you help me?

---

<div class="post-metadata">

**Author:** ![chrisk-dd](https://avatars.discourse-cdn.com/v4/letter/c/b5ac83/32.png) [@chrisk-dd](https://community.drivendata.org/u/chrisk-dd)\
**Post date:** [December 16, 2022, 4:14pm UTC](https://community.drivendata.org/t/failed-without-log/8296/2 "2022-12-16T16:14:01Z")

</div>

Hi Wenhaowang-  
We are investigating and will report back when we identify the cause.  
-Chris

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 4:16pm UTC](https://community.drivendata.org/t/failed-without-log/8296/3 "2022-12-16T16:16:18Z")

</div>

After running the [runtime](https://github.com/drivendataorg/meta-vsc-descriptor-runtime) locally, I find the reason may be that the 112G memory is not enough (It may need 150G memory). But I exactly follow the requirement by one second one feature. Could you increase the memory limitation to 200G?

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 4:18pm UTC](https://community.drivendata.org/t/failed-without-log/8296/4 "2022-12-16T16:18:12Z")

</div>

The “one second one feature” setting results in 323060 query features and 1267840 ref features.

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 4:20pm UTC](https://community.drivendata.org/t/failed-without-log/8296/5 "2022-12-16T16:20:04Z")

</div>

What is the number of features generated by the SSCD baseline? The number of features from mine seems to be very similar to the number of features from SSCD (less than a 1% difference).

---

<div class="post-metadata">

**Author:** ![chrisk-dd](https://avatars.discourse-cdn.com/v4/letter/c/b5ac83/32.png) [@chrisk-dd](https://community.drivendata.org/u/chrisk-dd)\
**Post date:** [December 16, 2022, 9:20pm UTC](https://community.drivendata.org/t/failed-without-log/8296/6 "2022-12-16T21:20:15Z")

</div>

The baseline uses 346,002 query features and 1,260,713 ref features. We don’t see memory problems with the baseline, but we reproduced that your submission does run out of memory. We are working to figure out the root cause of the memory issue and will report back when we have more information.

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 9:37pm UTC](https://community.drivendata.org/t/failed-without-log/8296/7 "2022-12-16T21:37:57Z")

</div>

Ok, thanks a lot!!! Hope it solved soon.

---

<div class="post-metadata">

**Author:** ![edpizzi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/edpizzi/32/1493_2.png) [@edpizzi](https://community.drivendata.org/u/edpizzi)\
**Post date:** [December 16, 2022, 10:43pm UTC](https://community.drivendata.org/t/failed-without-log/8296/8 "2022-12-16T22:43:10Z")

</div>

@wenhaowang How many dimensions per descriptor are you using?

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 10:54pm UTC](https://community.drivendata.org/t/failed-without-log/8296/10 "2022-12-16T22:54:35Z")

</div>

I use 512 dim. The same as yours

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 11:04pm UTC](https://community.drivendata.org/t/failed-without-log/8296/11 "2022-12-16T23:04:27Z")

</div>

Hi guys, I may find out the reason:  
Considering the code

```auto
def load_features(f, dataset: Optional[Dataset] = None):
    data = np.load(f, allow_pickle=False)
    video_ids = data["video_ids"]
    feats = data["features"]
    timestamps = data["timestamps"]

    ts_dims = len(timestamps.shape)
    if timestamps.shape[0] != feats.shape[0]:
        raise ValueError(
            f"Expected the same number of timestamps as features: got "
            f"{timestamps.shape[0]} timestamps for {feats.shape[0]} features"
        )
    if not (ts_dims == 1 or timestamps.shape[1:] == (2,)):
        print(f"timestamps.shape[1:]: {timestamps.shape[1:]}")
        print(f"timestamps.shape[1:] == [2]: {timestamps.shape[1:] == [2]}")
        raise ValueError(f"Unexpected timestamp shape. Got {timestamps.shape}")

    results = []
    for video_id, start, end in same_value_ranges(video_ids):
        video_id = format_video_id(video_id, dataset)
        results.append(
            VideoFeature(
                video_id=video_id,
                timestamps=timestamps[start:end],
                feature=feats[start:end, :],
            )
        )
    return results

```

It requires the names of queries are listed in sequence: i.e. Q200000, Q200000, Q200001, Q200001.

But mine is not arranged as this. Mine is something like Q200044, Q200000, Q200001, …

I think it is the problem, but I am not pretty sure.

---

<div class="post-metadata">

**Author:** ![chrisk-dd](https://avatars.discourse-cdn.com/v4/letter/c/b5ac83/32.png) [@chrisk-dd](https://community.drivendata.org/u/chrisk-dd)\
**Post date:** [December 16, 2022, 11:10pm UTC](https://community.drivendata.org/t/failed-without-log/8296/12 "2022-12-16T23:10:42Z")

</div>

Can you try submitting with sorted descriptors and see if it goes through? If that is the issue, we should be able to modify the eval script to handle this sorting automatically.

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 16, 2022, 11:18pm UTC](https://community.drivendata.org/t/failed-without-log/8296/13 "2022-12-16T23:18:18Z")

</div>

Yes. I am trying to do this.

---

<div class="post-metadata">

**Author:** ![edpizzi](https://yyz2.discourse-cdn.com/flex028/user_avatar/community.drivendata.org/edpizzi/32/1493_2.png) [@edpizzi](https://community.drivendata.org/u/edpizzi)\
**Post date:** [December 17, 2022, 3:34am UTC](https://community.drivendata.org/t/failed-without-log/8296/14 "2022-12-17T03:34:02Z")

</div>

Ah, I see. If you use the store\_features function in that file it will create features in the expected format.

That requires putting features into the VideoFeature format though.

Ed

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 17, 2022, 3:41am UTC](https://community.drivendata.org/t/failed-without-log/8296/15 "2022-12-17T03:41:39Z")

</div>

Haha. I write nearly all the code by myself and do not use that.

---

<div class="post-metadata">

**Author:** ![wenhaowang](https://avatars.discourse-cdn.com/v4/letter/w/b2d939/32.png) [@wenhaowang](https://community.drivendata.org/u/wenhaowang)\
**Post date:** [December 17, 2022, 3:55am UTC](https://community.drivendata.org/t/failed-without-log/8296/16 "2022-12-17T03:55:13Z")

</div>

@chrisk-dd It works. It is the sorting issue. Thanks.

Thanks too! @edpizzi