- Large Language Models (LLMs) often show poor metacognition, leading to overconfidence, hallucinations, and failure to recognize their own knowledge limits.
- The paper introduces Reinforcement Learning with Metacognitive Feedback (RLMF), a method that uses a model's self-judgments of performance to improve completion rankings during preference optimization and for selecting high-value training data.
- A second method, metacognitive data selection, uses similar self-judgments to outperform standard active learning by identifying more useful training examples.
- These innovations are applied to the task of faithful calibration (FC), aiming to align an LLM's expressed uncertainty with its actual intrinsic uncertainty.