Llama 3
Meta
partial detail 10 method fields published
Evaluation
FP8 inference efficiency (throughput-latency)
Throughput-latency trade-off of FP8 inference with Llama 3 405B in pre-fill and decoding stages (4,096 input tokens, 256 output tokens) vs two-machine BF16 inference
How it was run
Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.