Recovering logs from submissions

TissueNet: Detect Lesions in Cervical Biopsies
1

Is there a way to recover custom logs from submissions?
Below is a sample minimal code that times the processing of each test image and writes the result to the default logger (recovered with logger.getLogger()). The timings, however, do not appear in the execution logs recovered from the code execution status page. Is this on purpose, or should I use another Logger instance?

import pandas as pd
import os
import logging
import time
from random import randrange

if __name__=="__main__":
    logger = logging.getLogger()
    text_output = 'filename,0,1,2,3\n'
    df = pd.read_csv(os.path.join("data", "test_metadata.csv"))
    
    logger.info("Start of main loop")
    t_start = time.time()
    for filename in df.filename:
        
        # Do something
        prediction = randrange(4)
        
        line_output = filename + ',' + ','.join([str(int(ii==prediction)) for ii in range(4)]) + '\n'
        text_output += line_output
        t_end = time.time()
        logger.info(f"Processed {filename} in {t_end-t_start:.1f} seconds")
        t_start = t_end

    logger.info("Processing complete, printing on csv")
    with open("submission.csv", "w") as fo:
        fo.write(text_output)
2

I think you just forgot to configure your logger with a line like:
logging.basicConfig(level=logging.INFO)

You can test this locally with Docker to make sure you have a script that does what you expect, and then let us know if you see any differences between the local execution and what happens on the platform. Instructions to do so are here:

3

Thanks! I’ll try that and let you know if there are discrepancies in the logging behavior between executions in a local container or on the platform.

4

I’m using the logger as stated above, and I can see the logger output while my code is running, but sometimes it says ‘No execution log output’ when the job finishes (i.e. it has lost all of the logger output that was previously there). Please could you advise on what might be causing this.