Do Smaller, Full-Precision Models Outperform Quantized Code Models? | HackerNoon
The increase in inference time in higher precision models is mainly due to longer forward pass time rather than longer output generation time. Higher precision models take longer to compute.