Evaluating the effectiveness of multi-step automated source code refactoring using a large language model

Abstract

Topicality: modern large language models open up new opportunities for automating source code refactoring; however, most existing studies focus on individual refactoring operations, while the automated execution of sequences of such operations remains insufficiently studied. The aim of the article is to study the effectiveness of using a large language model for automated multi-step refactoring of Java program code. Objectives: to develop multi-step refactoring scenarios, conduct an experimental evaluation of their execution by the large language model, analyze the influence of the initial characteristics of the source code on the success of refactoring, compare the effectiveness of different scenarios and identify typical errors that occur during the execution of multi-step transformations. Methods: the effectiveness was assessed by the Maintainability Index, Cyclomatic Complexity, Cognitive Complexity, Lines of Code, the number of Code Smells, as well as successful compilation and unit test execution. Scientific novelty: lies in assessing the effectiveness of a large language model for performing multi-step refactoring, which involves a sequence of interrelated operations, as well as studying the influence of program code characteristics and the complexity of scenarios on the success of their automated execution. Practical significance: the results obtained can be used in development of intelligent automated refactoring tools, integration of large language models into software development environments, and improvement of source code quality assurance processes. Results: the experimental study showed that the large language model successfully performed multi-step refactoring scenarios, ensuring an improvement in the primary indicators of source code quality. Conclusions: it was established that the effectiveness of automated refactoring depends on both the initial characteristics of the source code and the complexity of the refactoring scenario, and the main limitations remain the execution of long sequences of interrelated operations and ensuring full functional equivalence of the program code.