Can We Know Whether a Profiler Is Accurate?
2 days ago
- #Java Profiling
- #Performance Analysis
- #Accuracy Assessment
- Sampling profilers for Java can be inaccurate due to the observer effect, where profiling alters performance behavior.
- Cycle-accurate simulation to determine accurate profiles is impractical due to the complexity of modern processors and runtimes.
- Mytkowicz et al. (2010) proposed checking profiler accuracy by inserting computations at the Java bytecode level, but this interacts unpredictably with compiler optimizations.
- A novel approach involves accurately slowing down programs to detect profiler inaccuracies by observing changes in time distribution.
- Experiments show that async-profiler and JFR are relatively accurate, while JProfiler and YourKit fail to correctly attribute runtime changes.
- The slowdown-based method provides a practical way to approximate ground truth and assess profiler accuracy.
- The paper details implementation, methodology, and findings, including cases where missing debug information causes inaccuracies.