GraalSP Profiles Logger: A Tool for Analyzing and Interpreting Predictions of the ML-Based Static Profilers

Program profiles provide information about the program’s behaviour during execution. Profiles include the number of branch executions in branching statements, context and number of function calls, context and number of virtual method calls, information on monitor locks and unlocks, and more. Profiles enable profile-guided optimizations including aggressive inlining, cache memory optimizations, and similar techniques which generate highly efficient code. Instrumentation-based profiling is a common technique for collecting precise profiles. However, the profile collection represents a significant overhead in software development, and in some environments, it is unfeasible (for example, in memory-constrained environments such as embedded systems or in real-time systems where extended execution time due to the additional instrumentation code is not acceptable). An alternative to dynamic profiling is static profiling. Static profilers predict profiles instead of collecting them. The current development of static profilers is mainly focused on using machine learning techniques for predicting branch execution probabilities. Modern static profilers use ensembles of random forests and deep neural networks for regression to predict the probabilities of the taken branch of a branching statement. The best current static profilers achieve execution time speedups of 5 to 7 per cent. Static profilers can make mispredictions as similar code segments may execute differently depending on inputs and various conditions. The impact of individual predictions can significantly affect the performance of the program. For instance, an incorrect prediction that the branch would not lead into a frequent loop body can result in a substantially slower program. To maximize performance, state-of-the-art static profilers employ instance weighting to prioritize critical cases. However, misprediction can still happen and it is necessary to efficiently identify and analyze them, to adjust and improve the ML model driving the static profiler. We have developed the GraalSP Profiles Logger (GraalSP-PLog), a tool for efficiently analyzing ML-based predictions made by the GraalSP static profiler which is a part of the Oracle GraalVM Native Image compiler. GraalSP-PLog runs the input program with GraalSP to log the profile predictions. As GraalSP employs a set of static heuristics that correct predictions of the ML model, GraalSP-PLog also captures this information. To collect ground truth values, GraalSP-PLog performs instrumentation profiling to dynamically collect program profiles. Combining static predictions and collected profiles, GraalSP-PLog generates a prediction report. The report facilitates the identification of performance-critical predictions, both accurate and inaccurate, and sorts nodes by their impact on overall program performance. The tool provides results in .csv format so that they can be easily opened on any operating system and sorted by any criteria...