🌲 Tree-Based AutoML Regression Framework

A modular, configuration-driven tree-based regression framework for tabular data.

It automates the full workflow from a single YAML file: data loading, exploratory data analysis, preprocessing, hyperparameter tuning with Optuna, model training, evaluation, and HTML reporting.

---

Overview

This project is designed to make regression experiments faster, cleaner, and more reproducible.

Instead of hardcoding parameters in Python scripts, you define your experiment in config.yaml and run the pipeline. The framework then handles preprocessing, model selection, tuning, evaluation, and reporting automatically.

It is especially useful for:

• benchmarking tree-based regression models
• testing tabular datasets quickly
• generating reproducible experiments
• comparing models with consistent preprocessing and cross-validation

---

Why tree-based?

The supported models are all tree-based or tree-ensemble methods:

• Random Forest — bagging over decision trees
• Histogram-based Gradient Boosting — boosting with histogram-based trees
• LightGBM — gradient boosting tree model
• XGBoost — gradient boosting tree model
• CatBoost — gradient boosting tree model

That makes the repository name more descriptive and immediately clear to readers.

---

Features

• One YAML, full control — change the data source, target column, model, tuning settings, and outputs without editing the code.
• Automatic preprocessing — missing values are handled intelligently, and categorical features are encoded automatically.
• Built-in regression models — Random Forest, Histogram-based Gradient Boosting, LightGBM, XGBoost, and CatBoost.
• Optuna hyperparameter tuning — supports robust search with repeated K-fold cross-validation.
• Single-model or multi-model mode — run one model or compare several models side by side.
• Interactive HTML reports — EDA, tuning history, evaluation plots, and comparison summaries.
• Easy to extend — add your own model to the registry with minimal changes.

---

Installation

Clone the repository and install the dependencies:

git clone https://github.com/GreenSmart-DSS/tree-based-automl-regression.git
cd tree-based-automl-regression
pip install -r requirements.txt

Base dependencies

These packages are required in all cases:

pandas
scikit-learn
optuna
plotly
pyyaml
openpyxl

Optional model libraries

Install these only if you want to use the corresponding models:

pip install lightgbm
pip install xgboost
pip install catboost

---

Quick Start

1) Prepare your data

Use a CSV or Excel file containing your features and a numeric target column.

2) Edit config.yaml

Example:

data:
  source: "data/housing.csv"
  target_column: "MedHouseVal"
  # features:
  #   - "MedInc"
  #   - "AveRooms"
  #   - "AveOccup"

model:
  name: "xgboost"

tuning:
  n_trials: 50
  cv:
    n_splits: 5
    n_repeats: 3

output:
  report_dir: "reports"

3) Run the pipeline

python run.py

4) Review the output

The generated reports will be saved in the reports/ directory.

---

Multi-Model Comparison

To compare several models automatically, use the names list in the configuration:

model:
  names:
    - "random_forest"
    - "xgboost"
    - "lightgbm"
    - "hist_gb"
    - "catboost"

In this mode, each model is tuned and evaluated separately, and a final comparison report is generated with metrics such as:

• RMSE
• MAE
• R²

The best-performing model is highlighted in the final comparison.

---

Supported Models

Config Key	Model	Library
random_forest	Random Forest Regressor	scikit-learn
hist_gb	Histogram-based Gradient Boosting	scikit-learn
lightgbm	LightGBM Regressor	lightgbm
xgboost	XGBoost Regressor	xgboost
catboost	CatBoost Regressor	catboost

You can extend the framework by adding new builders to the model registry.

---

Configuration Reference

data:
  source: "path/to/file.csv"       # .csv, .xlsx, .xls
  target_column: "target"
  features:                        # optional list of columns
    - "feat1"
    - "feat2"
  split:
    test_size: 0.2                 # hold-out fraction
    random_state: 42

model:
  name: "random_forest"            # single-model mode
  # OR
  names:                           # multi-model mode
    - "random_forest"
    - "xgboost"

tuning:
  n_trials: 30                     # Optuna trials per model
  cv:
    n_splits: 5                    # K-fold splits
    n_repeats: 3                   # repeated CV

output:
  report_dir: "reports"            # output folder for HTML reports

---

Output Reports

The framework generates standalone HTML reports with interactive charts.

EDA Report

Includes:

• dataset overview
• target distribution
• missing value summary
• correlation analysis
• categorical feature summaries

Tuning Report

Includes:

• Optuna optimization history
• parameter importance
• top trial results
• search summary

Evaluation Report

Includes:

• actual vs predicted plot
• residual analysis
• regression metrics
• feature importance

Comparison Report

Available in multi-model mode and includes:

• metrics table
• RMSE and R² comparison charts
• best model identification

---

Programmatic Use

You can also use the framework from another Python script:

from framework.pipeline import run_pipeline, compare_models

# Single model
metrics = run_pipeline(
    data_source="data.csv",
    target_column="price",
    model_name="lightgbm",
    n_trials=30
)

# Multiple models
results = compare_models(
    data_source="data.csv",
    target_column="price",
    model_names=["random_forest", "xgboost", "catboost"],
    n_trials=30
)

---

Project Structure

.
├── config.yaml                  # Experiment configuration
├── run.py                       # Main entry point
├── requirements.txt
├── framework/
│   ├── __init__.py
│   ├── config.py                # YAML loader
│   ├── data.py                  # Data loading and EDA report
│   ├── models.py                # Model registry
│   ├── tuner.py                 # Optuna tuning logic
│   ├── evaluation.py            # Metrics and evaluation report
│   └── pipeline.py              # Main orchestration layer
├── examples/
│   └── dummy_demo.py            # Example usage
└── reports/                     # Generated HTML reports

---

Notes

• Rows with missing target values are dropped automatically.
• Numeric features are imputed and scaled during preprocessing.
• Categorical features are One-Hot Encoded.
• Random seeds are fixed where possible to improve reproducibility.
• The framework is designed for regression problems on tabular data.

---

Extending the Framework

To add a new model:

1. Define a model builder function.
2. Register it in framework/models.py.
3. Add its hyperparameter search space.
4. Test it through config.yaml.

This keeps the framework modular and easy to maintain.

---

Example Use Cases

This framework can be used for:

• house price prediction
• agricultural yield estimation
• water demand forecasting
• environmental modeling
• business analytics regression tasks
• scientific tabular datasets

---

Citation

If you use this repository in academic work, please cite it as:

Morteza Khoshsimaie Chenar. (2026). Tree-Based AutoML Regression Framework. GitHub repository.

---

License

This project is released under the MIT License.
You are free to use, modify, and share it with attribution.

---

Acknowledgements

Built with:

• Python
• scikit-learn
• Optuna
• Plotly
• LightGBM
• XGBoost
• CatBoost

---

Contact

For questions, suggestions, or collaboration, feel free to open an issue or submit a pull request.

---

If you find this project useful, please consider starring the repository.