I told 3090 owners to skip MTP. I was very wrong.
Last time I told 3090 owners to skip MTP. Then I took my own advice and ran it on my 3090: +41%, the biggest speedup in the series, on the exact card I’d said to skip - while the same trick lost on an MI300X. The variable that decided it wasn’t bandwidth but the cost ratio from the 2023 speculative-decoding paper, the…
Read the full post →