What's the G6e instance size for evaluation?

Meta CRAG - MM Challenge 2025
#1

Hi orgainzer,
I saw the rule mentioned that:

All submissions will be run on a single G6e instance with a NVIDIA L40s GPU with 48GB of GPU memory on AWS.

I want to understand which size [1] of G6e instance is used ? g6e-xlarge? This information will help for our offine evaluation.

[1] Amazon EC2 G6e Instances | Amazon Web Services

Best
Fanyou

2 Likes
#2

Hi wufanyou.

I found the description below in Starter Kit repo.

  1. Hardware: Your code will run on an NVIDIA L40s GPU with 4 vCPUs, 32 GB RAM, and no internet access (HF_HUB_OFFLINE=1).

They specified vCPUs as 4, so I think the instance size is g6e-xlarge!

But there is no infomation about the storage.
Dear organizers, are there any storage limitations in the evaluation environment?

1 Like
#3

@cobochan : The instance we use is a g6e-2xlarge instance. And the disk storage available is 300GB per evaluation container.
You are right to note that 4vCPUs are available, even the instance type has 8vCPUs. We make 4 vCPUs available to the evaluation container, and the other 4 vCPUs might be used up by the background services running on the same instance.