Deterministic Programming with LLMs
3 days ago
- #LLMs
- #Deterministic Programming
- #Code Verification
- LLMs are changing the programming industry, with debates on ethics, best practices, and effective usage.
- Mathematicians use LLMs for proofs but rely on tools like Lean for verification due to LLM hallucinations.
- A team used ChatGPT and Aristotle to solve a misrecorded Erdős problem, combining LLM creativity with deterministic verification.
- In programming, deterministic tools like deployment scripts are preferred for reliability over manual or LLM-based methods.
- LLMs are not fully deterministic, making them unreliable for tasks requiring consistent execution, like preventing injection attacks.
- Code-checking code (lints, tests, type systems) ensures deterministic enforcement of coding standards, a task LLMs can help create but not reliably execute.
- For one-off tasks, LLMs are useful, but for repeated tasks requiring consistency, deterministic programs are necessary.