| 
            
            
            
            
              Why submission failed?
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            2
           | 
          
            547
           | 
          
            May 27, 2024
           | 
        
        
          | 
            
            
            
            
              Data quality V3: Inconsistency between answer and query_time
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            414
           | 
          
            May 27, 2024
           | 
        
        
          | 
            
            
            
            
              Oh, ops, it seems to be my submission failed abnormally "workflow shutdown with strategy: Stop"
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            1
           | 
          
            552
           | 
          
            May 27, 2024
           | 
        
        
          | 
            
            
            
            
              Issue with Submission
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            427
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              Temporary Service Outage because of Service Quotas on AWS
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            1
           | 
          
            488
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              Temporary Service Outage because of Service Quotas on AWS
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            434
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              Push Docker image error
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
           | 
          
            4
           | 
          
            754
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              The evaluation queue seems to be completely congested. Should the operations staff check for some timed-out tasks and kill them to clear the queue?
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            498
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
              
            
            
              Evaluation failed without any text message? So far, I haven't had any successful submissions in round 1b task1/2
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            1
           | 
          
            329
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              Evaluation timed out
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            2
           | 
          
            516
           | 
          
            May 26, 2024
           | 
        
        
          | 
            
            
            
            
              Does the evaluation support async await calls?
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            410
           | 
          
            May 25, 2024
           | 
        
        
          | 
            
            
            
            
              The openai interface cannot be used during evaluation? Why?
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            1
           | 
          
            838
           | 
          
            May 25, 2024
           | 
        
        
          | 
            
            
              
            
            
              Phase 2 launching!
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
           | 
          
            6
           | 
          
            2115
           | 
          
            May 25, 2024
           | 
        
        
          | 
            
            
            
            
              Submission with 500 Server Error
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
           | 
          
            5
           | 
          
            933
           | 
          
            May 24, 2024
           | 
        
        
          | 
            
            
            
            
              Inference failed without any useful message in debug-logs
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
              
                 
              
              
                 
              
           | 
          
            8
           | 
          
            863
           | 
          
            May 24, 2024
           | 
        
        
          | 
            
            
            
            
              Are LLama models of unsloth allowed?
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            499
           | 
          
            May 24, 2024
           | 
        
        
          | 
            
            
            
            
              Inference failed with vLLM Error: "Too large swap space"
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            2
           | 
          
            4175
           | 
          
            May 22, 2024
           | 
        
        
          | 
            
            
              
            
            
              Failed inference
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
           | 
          
            3
           | 
          
            567
           | 
          
            May 21, 2024
           | 
        
        
          | 
            
            
              
            
            
              Gitlab could not log in
            
            
           | 
          
              
                 
              
           | 
          
            1
           | 
          
            464
           | 
          
            May 21, 2024
           | 
        
        
          | 
            
            
              
            
            
              Couldn't instantiate the backend tokenizer
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            1
           | 
          
            641
           | 
          
            May 21, 2024
           | 
        
        
          | 
            
            
            
            
              Data Quality Collection V2, Task-1
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
              
                 
              
           | 
          
            12
           | 
          
            837
           | 
          
            May 21, 2024
           | 
        
        
          | 
            
            
              
            
            
              Evaluation succeeded but the results are not on the lead board
            
            
           | 
          
              
                 
              
              
                 
              
           | 
          
            2
           | 
          
            743
           | 
          
            May 20, 2024
           | 
        
        
          | 
            
            
            
            
              About Test Set Leakage in Round 1
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
              
                 
              
           | 
          
            3
           | 
          
            977
           | 
          
            May 20, 2024
           | 
        
        
          | 
            
            
            
            
              The evaluation was successful but is still in the submitted state
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            416
           | 
          
            May 19, 2024
           | 
        
        
          | 
            
            
            
            
              Push gitlab and cannot find issue
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
              
                 
              
              
                 
              
           | 
          
            8
           | 
          
            561
           | 
          
            May 18, 2024
           | 
        
        
          | 
            
            
            
            
              Still seen no issue after submit(solved)
            
            
           | 
          
              
                 
              
           | 
          
            1
           | 
          
            323
           | 
          
            May 18, 2024
           | 
        
        
          | 
            
            
            
            
              Inference failed without reporting logs, why is this?
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            321
           | 
          
            May 18, 2024
           | 
        
        
          | 
            
            
            
            
              Added tag with "submission-v" prefix but no evaluation issue created
            
            
           | 
          
              
                 
              
              
                 
              
              
                 
              
              
                 
              
              
                 
              
           | 
          
            7
           | 
          
            578
           | 
          
            May 18, 2024
           | 
        
        
          | 
            
            
              
            
            
              Submissions not launched
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            281
           | 
          
            May 17, 2024
           | 
        
        
          | 
            
            
            
            
              Expect to return a message that stating whether it was a timeout problem
            
            
           | 
          
              
                 
              
           | 
          
            0
           | 
          
            399
           | 
          
            May 17, 2024
           |