From Dev.to
Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost
I rented one RTX 3090 at $0.259/hour, served Qwen2.5-7B-Instruct with vLLM 0.27.1 and default flags, and kept it busy for 60 seconds at 1, 4, 16 and 64 requests at once. Each request had a prompt of a...