AutoFullStack is a multi-agent framework for generating full-stack management systems from natural language project requirements. Given a project name and description, it plans the project, generates backend and frontend code, and can place the generated project into an execution environment for build and runtime checks.
The framework is mainly designed for research on project-level software generation. It includes an incremental workflow, a waterfall-style baseline, and an incremental variant without self-refinement, so that different generation strategies can be compared under the same project scaffold and benchmark tasks.
AutoFullStack/
|-- auto_full_stack/ # Core implementation of the framework
| |-- workflows/ # Planning, development, operations, and baseline workflows
| |-- workflows/agent/ # Agent roles used by the workflows
| |-- workflows/prompt/ # Prompt templates for planning and code generation
| |-- templates/ # Full-stack project scaffold and configuration templates
| |-- common/ # Shared configuration, logging, workspace, and LLM utilities
| |-- utils/ # File, database, Redis, prompt, and report utilities
| |-- app.py # Entry point for running a single project generation task
| `-- benchmark_runner.py # Batch runner for benchmark tasks
|-- data/ # Benchmark task descriptions
|-- experiment_results/ # Generation and evaluation records for three runs
|-- evaluation/
| |-- scripts/ # Analysis scripts for the recorded experiments
| `-- artifacts/ # Tables, figures, and supporting data
|-- .env.example # Example environment configuration
|-- requirements.txt # Python dependencies
`-- README.md
AutoFullStack was developed and tested with Python 3.10.18 in a Conda environment. To run the framework, the following software and services are required:
- Python 3.10
- An OpenAI-compatible LLM API endpoint
- Java 17
- Maven
- MySQL 8.x
- Redis
Java and Maven are required because the development workflow builds and tests the generated Spring Boot backend during code generation. MySQL is also used during generation: AutoFullStack creates project-specific databases, executes generated SQL scripts, and runs backend tests against the database. The default MySQL connection is localhost:3306 with user root and password root, and the account must be allowed to create and drop databases. Redis is used to store workflow status, project namespaces, and process information. The default Redis connection is localhost:6379 without a password.
Generated projects use a Spring Boot backend and a Vue 2 frontend. Running a generated project requires:
- Java 17 and Maven for the backend
- Node.js and npm compatible with Vue CLI 4 for the frontend
- MySQL and Redis services
By default, generated projects use backend port 8080, frontend port 80, MySQL port 3306, and Redis port 6379.
Create a Python environment and install the required packages:
conda create -n AutoFullStack python=3.10
conda activate AutoFullStack
pip install -r requirements.txtAfter installation, you can check whether the Python entry point is available:
python -m auto_full_stack.app --helpMake sure Java, Maven, MySQL, Redis, Node.js, and npm are also installed and available from the command line before running a full generation workflow.
Create a local environment file from the example:
cp .env.example .envOn Windows PowerShell, use:
Copy-Item .env.example .envThen edit .env and set the LLM configuration:
LLM_API_KEY="your-api-key"
LLM_BASE_URL="https://api.openai.com/v1"
LLM_MODEL="your-model-name"LLM_BASE_URL can point to any OpenAI-compatible API endpoint. LangSmith variables in .env.example are optional and are only needed if you want to trace LangChain calls.
AutoFullStack currently assumes local MySQL and Redis services with the following defaults:
MySQL: localhost:3306, user=root, password=root
Redis: localhost:6379, no password
The MySQL account must have permission to create and drop databases because each generated project uses its own project-specific database.
Use auto_full_stack.app to run AutoFullStack on one project requirement:
python -m auto_full_stack.app \
--project-name "Personal Contact Manager" \
--description "Please help me create a personal contact management system. This system includes a contact management module. Contact management enables users to add, edit, delete and view contact information." \
--approach incrementalThe --approach option supports three workflow variants:
incremental
incremental_no_self_refinement
waterfall
If no arguments are provided, the script runs a default example project with the incremental approach.
Generated files are written to:
workspace/<project_namespace>/
The namespace is generated automatically from the project name. During a full run, AutoFullStack performs planning, code generation, backend verification, and operations steps. The operations step builds and starts the generated backend and frontend, so the process may keep running while the generated services are active.
By default, a generated project is written to:
workspace/<project_namespace>/
The project namespace is generated automatically from the project name. A generated project has the following main structure:
workspace/<project_namespace>/
|-- backend/ # Generated Spring Boot backend project
|-- frontend/ # Generated Vue 2 frontend project
|-- sql/ # Base SQL scripts and generated module SQL scripts
|-- resources/ # Planning and development artifacts
|-- logs/ # Runtime logs created by the operations workflow
|-- start.bat
`-- start.sh
The backend/ directory contains a Maven-based Spring Boot project, and the frontend/ directory contains a Vue 2 project built with Vue CLI. The sql/ directory includes base initialization scripts such as init.sql and quartz.sql, together with generated SQL scripts for project-specific modules. The resources/ directory stores intermediate artifacts produced during planning and development.
When auto_full_stack.app is run with a complete workflow, the operations step builds and starts the generated backend and frontend automatically. To run an already generated project manually, first make sure MySQL and Redis are running and that the project database has been initialized from the scripts in sql/.
For the backend:
cd workspace/<project_namespace>/backend
mvn clean test
mvn package -DskipTests
java -Dfile.encoding=UTF-8 -Dconsole.encoding=UTF-8 -jar admin/target/admin.jarFor the frontend:
cd workspace/<project_namespace>/frontend
npm install
npm run devThe database name is the same as <project_namespace>. In a full AutoFullStack run, the workflow initializes the database and executes the SQL scripts automatically. If you move or rerun a generated project in a fresh environment, import the SQL scripts in sql/, including quartz.sql, init.sql, and the generated create_tables_inc_*.sql and insert_menu_inc_*.sql files.
By default, the backend runs at:
http://localhost:8080
The frontend runs at:
http://localhost:80
If port 80 is unavailable or requires elevated permissions, change the frontend port in frontend/vue.config.js.
The data/ directory contains 30 benchmark tasks used for project-level full-stack generation:
data/simple_dataset.json # 10 simple tasks
data/medium_dataset.json # 10 medium tasks
data/complex_dataset.json # 10 complex tasks
Each task contains a project name and a natural language project description. These descriptions are used as inputs to AutoFullStack.
auto_full_stack.benchmark_runner runs benchmark tasks in batch mode. It supports the incremental workflow, the waterfall baseline, and the incremental workflow without self-refinement:
incremental
incremental_no_self_refinement
waterfall
Run one simple task with the incremental workflow:
python -m auto_full_stack.benchmark_runner --approach incremental --complexity simple --limit 1Run all tasks for one approach:
python -m auto_full_stack.benchmark_runner --approach incremental --complexity allRun all approaches on all benchmark tasks:
python -m auto_full_stack.benchmark_runner --approach all --complexity allRun a specific project by ID:
python -m auto_full_stack.benchmark_runner --approach incremental --complexity complex --project-id proj_complex_02By default, generated benchmark projects are written under:
experiments/<approach>/<complexity>/<project_namespace>/
This location can be changed with --workspace-base.
The batch runner writes summary CSV files to:
experiment_results/
For unattended batch execution, benchmark_runner.py runs the planning and development stages and skips the operations workflow. This means it records generation statistics such as status, module-level verification results, token usage, elapsed time, and generated lines of code, but it does not automatically perform the final backend/frontend runtime check for every generated project.
The records accompanying the revised paper are organized into three generation and evaluation runs:
experiment_results/
|-- run_1/
| |-- benchmark_results_*.csv # Three files, one per AutoFullStack variant
| |-- eval_e2e_executability.csv
| `-- eval_functional_correctness.csv
|-- run_2/ # Same file types
`-- run_3/ # Same file types
Each run contains generation statistics for 30 tasks under each of the Incremental, Waterfall, and Incremental-NoSelfRefinement variants. The evaluation CSVs also contain the MetaGPT comparison on the 10 Simple tasks.
The benchmark_results_*.csv files are produced by benchmark_runner.py. They record generation status, module-level verification results, token usage, elapsed time, and generated lines of code. Generation-stage verification fields, including CompilePassHeuristic, are separate from the final project-level evaluation.
eval_e2e_executability.csv records the final backend and frontend executability outcomes in the Backend and Frontend columns, using Pass or Failure. The evaluation protocol requires successful build/compilation, startup, and runtime checks; project-level success requires both components to pass.
eval_functional_correctness.csv records each module's final score from 0 to 5 and evaluator observations. Functional evaluation considers the requested backend APIs, frontend pages, and frontend-backend interactions against the original task requirements, following the scoring rubric in the paper.
For the AutoFullStack experiments reported in the paper, the generation model was DeepSeek-V3.2 with a temperature of 0.7. Each task was generated and evaluated three times under each method. Set LLM_BASE_URL and LLM_MODEL for an API endpoint serving this model; the required model identifier depends on the provider. The example environment configuration is intended for general framework use.
To repeat generation, run each AutoFullStack variant separately for each repetition using the batch commands above. Use a separate --workspace-base for each repetition and retain its timestamped benchmark CSV. The runner writes these CSVs directly to experiment_results/; it does not automatically organize them into run_1/, run_2/, and run_3/.
After generation, perform the final component checks and human functional evaluation separately, and record the outcomes, module scores, and observations in the corresponding evaluation CSVs. The batch runner skips the operations workflow, and neither it nor the analysis scripts automatically populate these evaluation records. The batch commands above apply to the AutoFullStack variants; they do not run MetaGPT.
The analysis scripts in evaluation/scripts/ read the supplied CSVs without invoking LLM generation or running the generated applications. The analysis uses NumPy and Matplotlib. To regenerate the tables, figures, and statistical summaries, run from the repository root:
python evaluation/scripts/generate_experiment_artifacts.pyBy default, the script reads all three runs and writes the following output structure:
evaluation/artifacts/
|-- combined/ # Combined results and paired statistical summaries
|-- runs/ # Separate outputs for run_1, run_2, and run_3
`-- run_comparison/ # Cross-run summaries and figures
Each scope separates LaTeX tables/, publication PDF figures/, full-precision CSV data/, and formatted preview/ files. Artifacts for the Simple-only MetaGPT quality comparison use the external_simple_ filename prefix.
Use --runs to select existing runs for analysis, --tables-only to omit figure rendering, or --output-dir to change the output destination. The analysis targets the supplied three-run records through the explicit benchmark filename mapping in evaluation/scripts/experiment_data.py. Newly generated, timestamped outputs require separate organization and an updated filename mapping before analysis.
AutoFullStack is intended as a research prototype for project-level full-stack software generation. The generated code and benchmark results may vary across runs because the framework relies on LLM outputs, external API availability, and the local execution environment.
The current implementation assumes local MySQL and Redis services with default credentials and ports. These settings are convenient for controlled experiments but should be adjusted before using the generated projects in other environments.
The benchmark tasks mainly cover management-style full-stack applications based on a shared project scaffold. They are not intended to represent all possible software development scenarios.
The project scaffold used by AutoFullStack is adapted from RuoYi-Vue, which is distributed under the MIT License. Its source, modification scope, copyright notice, and license text are documented in THIRD_PARTY_NOTICES.md.