From Dev.to
My benchmark reported 4,393,158 tokens per second
My API benchmark once reported 4,393,158 tokens per second. The useful result was finding out what that number actually measured. The harness timed a streaming response and calculated throughput like...