This is the release note of v4.4.0.

Highlights

In addition to new features, bug fixes, and improvements in documentation and testing, version 4.4 introduces a new tool called the Optuna MCP Server.

Optuna MCP Server

The Optuna MCP server can be accessed by any MCP client via uv — for instance, with Claude Desktop, simply add the following configuration to your MCP server settings file. Of course, other LLM clients like VSCode or Cline can also be used similarly. You can also access it via Docker. If you want to persist the results, you can use the — storage option. For details, please refer to the repository.

{
  "mcpServers": {
    … (Other MCP Servers' settings)
    "Optuna": {
      "command": "uvx",
      "args": [
        "optuna-mcp"
      ]
    }
  }
}

Gaussian Process-Based Multi-objective Optimization

Optuna’s GPSampler, introduced in version 3.6, offers superior speed and performance compared to existing Bayesian optimization frameworks, particularly when handling objective functions with discrete variables. In Optuna v4.4, we have extended this GPSampler to support multi-objective optimization problems. The applications of multi-objective optimization are broad, and the new multi-objective capabilities introduced in this GPSampler are expected to find applications in fields such as material design, experimental design problems, and high-cost hyperparameter optimization.

GPSampler can be easily integrated into your program and performs well against the existing BoTorchSampler. We encourage you to try it out with your multi-objective optimization problems.

:::python
sampler = optuna.samplers.GPSampler()
study = optuna.create_study(directions=["minimize", "minimize"], sampler=sampler)

New Features in OptunaHub

During the development period of Optuna v4.4, several new features were also introduced to OptunaHub, the feature-sharing platform for Optuna:

• A sampler utilizing Google Vizier is now newly available.
• The CMA-ES-based sampler with restart strategy, which was previously part of the Optuna core, has been migrated to OptunaHub, making it simpler and more user-friendly.
• A benchmark problem solving aircraft design as a black-box optimization task has been added, further enhancing the convenience of algorithm development using OptunaHub.
• A visualization feature has also been added, allowing users to see how the acquisition function of the default TPESampler evolves as trials progress.
• A novel mutation operation has been added to the MOEA/D evolutionary computation algorithm for multi-objective optimization.

Vizier sampler performance

TPE acquisition visualizer

Breaking Changes

• Update consider_prior Behavior and Remove Support for False (#6007)
• Remove restart_strategy and inc_popsize to simplify CmaEsSampler (#6025)
• Make all arguments of TPESampler keyword-only (#6041)

New Features

• Add a module to preprocess solutions for hypervolume improvement calculation (#6039)
• Add AcquisitionFuncParams for LogEHVI (#6052)
• Support Multi-Objective Optimization GPSampler (#6069)
• Add n_recent_trials to plot_timeline (#6110, thanks @msdsm!)

Enhancements

• Adapt TYPE_CHECKING of samplers/_gp/sampler.py (#6059)
• Avoid deepcopy in _tell_with_warning (#6079)
• Add _compute_3d for hypervolume computation (#6112, thanks @shmurai!)
• Improve performance of plot_hypervolume_history (#6115, thanks @shmurai!)
• add deprecated/removed version specification to calls of convert_positional_args (#6117, thanks @shmurai!)
• Optimize Study.best_trial performance by avoiding unnecessary deep copy (#6119, thanks @msdsm!)
• Refactor and speed up HV3D (#6124)
• Add assume_pareto for hv calculation in _calculate_weights_below_for_multi_objective (#6129)

Bug Fixes

• Update vsbx (#6033, thanks @hrntsm!)
• Fix request.values in OptunaStorageProxyService (#6044, thanks @hitsgub!)
• Fix a bug in distributed optimization using NSGA-II/III (#6066, thanks @leevers!)
• Fix: fetch all trials in BruteForceSampler for HyperbandPruner (#6107)

Documentation

• Add Pycma Example (https://github.com/optuna/optuna-integration/pull/226, thanks @ParagEkbote!)
• Add SHAP Example (https://github.com/optuna/optuna-integration/pull/227, thanks @ParagEkbote!)
• Document Behavior of optuna.pruners.MedianPruner and optuna.pruners.PatientPruner (#6055, thanks @ParagEkbote!)
• Change the link of tutorial docs of optunahub (#6063, thanks @fusawa-yugo!)
• Update the documentation string of GPSampler (#6081)
• Add a warning about the combination of gRPC Proxy and Journal Storage (#6097)
• Cosmetic fix to the terminator documents (#6100)
• Note in docstring that heartbeat mechanism is experimental (#6111, thanks @lan496!)
• Update docstrings of _get_best_trial to follow coding conventions (#6122)

Examples

• Add Example for Comet (https://github.com/optuna/optuna-examples/pull/305, thanks @ParagEkbote!)
• Adding an OpenML example (https://github.com/optuna/optuna-examples/pull/310, thanks @SubhadityaMukherjee!)
• Add workflow dispatch (https://github.com/optuna/optuna-examples/pull/311)
• Update PyTorch Checkpoint Example using tempfile (https://github.com/optuna/optuna-examples/pull/313, thanks @ParagEkbote!)
• [hotfix] Fix Dask-ML example by adding the version constraint on numpy (https://github.com/optuna/optuna-examples/pull/315)
• Setup Pre-Commit (https://github.com/optuna/optuna-examples/pull/316, thanks @ParagEkbote!)
• Remove Python 3.9 from haiku CI (https://github.com/optuna/optuna-examples/pull/318)
• Add a transformers example (https://github.com/optuna/optuna-examples/pull/322, thanks @ParagEkbote!)
• Add transformer item to RAEDME.md (https://github.com/optuna/optuna-examples/pull/323)
• Remove version constraints of tensorflow and numpy (https://github.com/optuna/optuna-examples/pull/324)
• Update pre-commit hooks (https://github.com/optuna/optuna-examples/pull/326, thanks @lan496!)
• Add preferential optimization picture (https://github.com/optuna/optuna-examples/pull/327, thanks @milkcoffeen!)

Tests

• Add float precision tests for storages (#6040)
• Refactor test_base_gasampler.py (#6104)
• chore: run tests for importance only with in-memory (#6109)
• Improve test cases for n_recent_trials of plot_timeline (follow-up [#6110]) (#6116)
• Performance optimization for test_study.py by removing redundancy (#6120)

Code Fixes

• Optional mypy check (#6028)
• Update Type-Checking for optuna/_experimental.py (#6045, thanks @ParagEkbote!)
• Update Type-Checking for optuna/importance/_base.py (#6046, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_convert_positional_args.py (#6050, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_deprecated.py (#6051, thanks @ParagEkbote!)
• Update Type-Checking for optuna/_gp/gp.py (#6053, thanks @ParagEkbote!)
• Add validate eta in sbx (#6056, thanks @hrntsm!)
• Remove CmaEsAttrKeys and _attr_keys for Simplification (#6068)
• Replace np.isnan with math.isnan (#6080)
• Refactor warning handling of _tell_with_warning (#6082)
• Implement Type-Checking for optuna/distributions.py (#6086, thanks @AdrianStrymer!)
• Update TYPE_CHECKING for optuna/_gp/gp.py (#6090, thanks @Samarthi!)
• Support mypy 1.16.0 (#6102)
• Emit ExperimentalWarning if heartbeat is enabled (#6106, thanks @lan496!)
• Simplify tuple return in optuna/visualization/_terminator_improvement.py (#6139, thanks @Prashantdhaka23!)
• Refactor return standardization in optim_mixed.py (#6140, thanks @Ajay-Satish-01!)
• Simplify tuple return in test_trial.py (#6141, thanks @saishreyakumar!)
• Refactor return statement style in optuna/storages/_rdb/models.py for consistency among the codebase (#6143, thanks @Shubham05122002!)

Continuous Integration

• Add type ignore in wandb (https://github.com/optuna/optuna-integration/pull/228)
• Fix to prevent daily checks-optional CI on the fork repositories (#6103)
• Fix CI (#6137)
• [hotfix] Add version constraint on blackdoc (#6145)

Other

• Bump up version number to v4.4.0.dev (https://github.com/optuna/optuna-integration/pull/220)
• Add pre commit config (https://github.com/optuna/optuna-integration/pull/231, thanks @milkcoffeen!)
• Bump up version number to 4.4.0 (https://github.com/optuna/optuna-integration/pull/236)
• Bump up version to 4.4.0dev (#6038)
• Update the news section in README.md (#6049)
• Fix README for v5 roadmap (#6094)
• Update pre-commit hooks (#6108, thanks @lan496!)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@AdrianStrymer, @Ajay-Satish-01, @Alnusjaponica, @Copilot, @HideakiImamura, @ParagEkbote, @Prashantdhaka23, @Samarthi, @Shubham05122002, @SubhadityaMukherjee, @c-bata, @contramundum53, @copilot-pull-request-reviewer[bot], @fusawa-yugo, @gen740, @himkt, @hitsgub, @hrntsm, @kAIto47802, @lan496, @leevers, @milkcoffeen, @msdsm, @nabenabe0928, @not522, @nzw0301, @saishreyakumar, @sawa3030, @shmurai, @toshihikoyanase, @y0z