Press Space to continue
A rack handled 6.77T tokens but p95 suffers from a flaky tray
Press Space to continue
in the last 24 hours this rack - handled 6.77T tokens - p50 TTFT was 1.16s - p50 TPS was 143 unfortunately p95 isn't looking great because one of the trays is flakey but as soon as we get it replaced...this is gonna be amazing
dax@thdxr6d ago
we had some fun news to share but got side tracked we did our first test run on a new GPU rack with our new deepseek inference setup the price/perf we're getting is better than every single vendor that we've found. and there's still more optimization to do