There is now a significant amount of effort, both in academia and industry, to build Software Engineering (SE) Agents. These LLM-powered systems go beyond simple code generation by interacting with the development environment to explore project structure, install software, examine logs, and run tests. However, collecting trajectories for in-context learning or training via reinforcement learning suffers from both GPU and CPU bottlenecks, particularly when running commands (like building or testing large projects) that can take a long time. Consequently, SE agent training is currently limited in its ability to comprehensively explore the trajectory search space or employ more sophisticated RL techniques, such as beam search or MCTS.

To address these challenges, this project proposes to build a high-performance training environment for SE agents that supports state sharing and checkpointing, branching states, and bypassing containerization when safe. This will (1) address performance bottlenecks in trajectory collection and training, (2) enable advanced RL techniques that go beyond simple trajectory sampling, and (3) facilitate training with custom development tools that take a long time to run. In addition, (4) we will develop an SE agent benchmark specifically for large Java projects. By gathering a large corpus of SE agent trajectories on both Python and Java tasks and rigorously benchmarking performance using state-of-the-art agent frameworks, we expect to develop SE agents that can solve harder programming tasks for both languages. This will directly benefit Oracle and the wider community by making SE agents more efficient and effective at software development.