Researchers at MIT and the MIT-IBM Watson AI Lab have developed a new calibration method for large language models (LLMs) called “Thermometer.” This technique addresses the issue of AI models being overly confident in their wrong answers and underconfident in correct ones. Unlike traditional calibration methods, Thermometer is efficient and preserves model accuracy, making it effective across various tasks without requiring power-intensive computation.
Thermometer works by implementing a smaller, auxiliary model on top of the LLM to predict the necessary “temperature” that aligns the model’s confidence with its accuracy. This approach eliminates the need for multiple training runs and uses minimal labeled data, allowing the calibrated LLM to generalize well across new tasks.
This technique’s efficiency and versatility could prevent the deployment of overconfident AI models in critical applications, ultimately aiding users in identifying when to trust AI-generated predictions. The researchers aim to adapt Thermometer for more complex tasks and larger LLMs, further enhancing its applicability and effectiveness.