Did somebody try deploy three llama3-8b on four T4?

Meta Comprehensive RAG Benchmark: KDD Cup 2-9d1937
#1

Did somebody try deploy three llama3-8b on four T4? I wonder if it’s possible to use three llama3-8b models

#2

How large is each llama3-8b model? If it is 16GB each, it’s possible.

#3

I tested it on my machine, and each model is about 20GB, so I wonder if it can be deployed online. :face_with_raised_eyebrow:

#4

There are 4 T4 cards. But I see that the total GPU memory is < 60GB.