Automated summarization of software documents: an LLM-based multi-agent approach

Large Language Models (LLMs) and LLM-based Multi-Agent Systems (MAS) are revolutionizing software engineering (SE) by advancing automation, decision-making, and knowledge processing. Their recent application to SE tasks has already shown promising results. In this paper, we focus on summarization as a key application area. We present Metagente, an LLM-based MAS designed to generate concise and accurate summaries of software documentation. Metagente employs a Teacher–Student architecture where multiple LLM agents collaborate to enhance relevance and precision of produced summaries. An empirical evaluation on real-world datasets demonstrates Metagente ’s effectiveness in streamlining workflows, outperforming the considered baselines. The evaluation provides evidence that Metagente improves summarization for requirement analysis and technical documentation. Moreover, we also demonstrate that compared to a set of single, independent LLMs, the multi-agent architecture is meaningful and beneficial to the summarization of software documents. Our findings underscore the transformative potential of these technologies in SE, while identifying challenges and future research directions for their seamless integration.