API Reference

RDBStorage(database_path::String, database_name::String)

Storage class for RDB backends.

Arguments

• database_path::String: Path to the directory where the database is stored.
• database_name::String: Name of the database file.

InMemoryStorage()

Storage class that stores data in memory of the running process.

get_all_study_names(storage::BaseStorage)

returns all study names stored in the given storage.

Arguments

• storage::BaseStorage: Storage backend to query.

Returns

• Vector{String}: List of study names.

FileSystemArtifactStore(path::String)

Data structure for a file system based artifact store.

Arguments

• path::String: Path to the directory where artifacts are stored.

ArtifactMeta

Data structure containing metadata for an artifact.

RandomSampler(seed=nothing::Union{Nothing,Integer})

An independent sampler that samples randomly. For further information see the RandomSampler in the Optuna python documentation.

Arguments

• seed::Union{Nothing,Integer}=nothing: Seed for the random number generator.

TPESampler(; ...)

Sampler using TPE (Tree-structured Parzen Estimator) algorithm. For further information see the TPESampler in the Optuna python documentation.

Keywords

• consider_prior::Bool (default=true).
• prior_weight::Float64 (default=1.0).
• consider_magic_clip::Bool (default=true).
• consider_endpoints::Bool (default=false).
• n_startup_trials::Integer (default=10).
• n_ei_candidates::Integer (default=24).
• seed::Union{Nothing,Integer}: Seed for the random number generator (default=nothing).
• multivariate::Bool (default=false).
• group::Bool (default=false).
• warn_independent_sampling::Bool (default=true).
• constant_liar::Bool (default=false).

GPSampler(; ...)

Sampler using Gaussian process-based Bayesian optimization. For further information see the GPSampler in the Optuna python documentation.

Keywords

• seed::Union{Nothing,Integer}: Seed for the random number generator (default=nothing).
• independent_sampler::Union{Nothing,BaseSampler}: Sampler used for independent sampling when needed (default=nothing).
• n_startup_trials::Int: Number of initial trials before Gaussian process-based optimization is used (default=10).
• deterministic_objective::Bool: Whether the objective function is assumed to be deterministic (default=false).
• warn_independent_sampling::Bool: Whether to show warnings when independent sampling is performed (default=true).

CmaEsSampler()

A sampler using cmaes as the backend.

For further information see the CmaEsSampler in the Optuna python documentation.

NSGAIISampler()

Multi-objective sampler using the NSGA-II algorithm.

For further information see the NSGAIISampler in the Optuna python documentation.

NSGAIIISampler()

Multi-objective sampler using the NSGA-III algorithm.

For further information see the NSGAIIISampler in the Optuna python documentation.

GridSampler(search_space, seed)

Sampler using grid search. For further information see the GridSampler in the Optuna python documentation.

Arguments

• search_space::Dict{String, Vector}
• seed::Union{Nothing,Integer}=nothing: Seed for the random number generator.

QMCSampler(; ...)

A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. For further information see the QMCSampler in the Optuna python documentation.

Keywords

• qmc_type::String (default="sobol"): Type of QMC sequence to use (for example, "sobol" or "halton").
• scramble::Bool (default=false): Whether to scramble the QMC sequence to improve uniformity.
• seed::Union{Nothing,Integer}: Seed for the random number generator (default=nothing).
• independent_sampler::Union{Nothing,BaseSampler}: Independent sampler to use for parameters or cases not handled by the QMC sequence (default=nothing).
• warn_asynchronous_seeding::Bool (default=true): Whether to emit a warning when using a fixed seed in asynchronous optimization.
• warn_independent_sampling::Bool (default=true): Whether to emit a warning when falling back to the independent sampler.

BruteForceSampler(seed, avoid_premature_stop)

Sampler using brute force.

This sampler performs exhaustive search on the defined search space. For further information see the BruteForceSampler in the Optuna python documentation.

Arguments

• seed::Union{Nothing,Integer}=nothing: Seed for the random number generator.
• avoid_premature_stop::Bool=false: If true, avoids stopping trials prematurely.

PartialFixedSampler(fixed_params, base_sampler)

Sampler with partially fixed parameters.

For further information see the PartialFixedSampler in the Optuna python documentation.

Arguments

• fixed_params::Dict{String,Vector}: Dictionary of parameter names to fixed values. Parameters listed here will be fixed to the given values during sampling.
• base_sampler::BaseSampler: Underlying sampler used to sample parameters that are not fixed in fixed_params.

MedianPruner(n_startup_trials::Int=5,
    n_warmup_steps::Int=0,
    interval_steps::Int=1;
    n_min_trials::Int=1)

Pruner using the median stopping rule. Prune if the trial's best intermediate result is worse than median of intermediate results of previous trials at the same step.

Arguments

• n_startup_trials::Int=5: Pruning is disabled until the given number of trials finish in the same study.
• n_warmup_steps::Int=0: Pruning is disabled until the trial exceeds the given number of step. Note that this feature assumes that step starts at zero.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported.
• n_min_trials::Int=1: Minimum number of reported trial results at a step to judge whether to prune. If the number of reported intermediate values from all trials at the current step is less than n_min_trials, the trial will not be pruned.

NopPruner()

Pruner which never prunes trials.

PatientPruner(wrapped_pruner::Union{BasePruner,Nothing},
    patience::Int;
    min_delta::Float64=0.0)

Pruner which wraps another pruner with tolerance. This pruner monitors intermediate values in a trial and prunes the trial if the improvement in the intermediate values after a patience period is less than a threshold.

Arguments

• wrapped_pruner::Union{BasePruner,Nothing}: Wrapped pruner to perform pruning when PatientPruner allows a trial to be pruned. If it is nothing, this pruner is equivalent to early-stopping taken the intermediate values in the individual trial.
• patience::Int: Pruning is disabled until the objective doesn't improve for patience consecutive steps.
• min_delta::Float64=0.0: Tolerance value to check whether or not the objective improves. This value should be non-negative.

PercentilePruner(percentile::Float64,
    n_startup_trials::Int=5,
    n_warmup_steps::Int=0,
    interval_steps::Int=1;
    n_min_trials::Int=1)

Pruner to keep the specified percentile of the trials. Prune if the best intermediate value is in the bottom percentile among trials at the same step.

Arguments

• percentile::Float64: Percentile which must be between 0 and 100 inclusive (e.g., When given 25.0, top of 25th percentile trials are kept).
• n_startup_trials::Int=5: Pruning is disabled until the given number of trials finish in the same study.
• n_warmup_steps::Int=0: Pruning is disabled until the trial exceeds the given number of step. Note that this feature assumes that step starts at zero.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported. Value must be at least 1.
• n_min_trials::Int=1: Minimum number of reported trial results at a step to judge whether to prune. If the number of reported intermediate values from all trials at the current step is less than n_min_trials, the trial will not be pruned.

SuccessiveHalvingPruner(;
    min_resource::Union{String,Int}="auto",
    reduction_factor::Int=4,
    min_early_stopping_rate::Int=0,
    bootstrap_count::Int=0)

Pruner using Asynchronous Successive Halving Algorithm.

Arguments

• min_resource::Union{String,Int}="auto": A parameter for specifying the minimum resource allocated to a trial (in the paper this parameter is referred to as r). This parameter defaults to "auto" where the value is determined based on a heuristic that looks at the number of required steps for the first trial to complete.
• reduction_factor::Int=4: A parameter for specifying reduction factor of promotable trials (in the paper this parameter is referred to as η). At the completion point of each rung, about 1/reduction_factor trials will be promoted.
• min_early_stopping_rate::Int=0: A parameter for specifying the minimum early-stopping rate (in the paper this parameter is referred to as s).
• bootstrap_count::Int=0: Minimum number of trials that need to complete a rung before any trial is considered for promotion into the next rung.

HyperbandPruner(;
    min_resource::Int=1,
    max_resource::Union{String,Int}="auto",
    reduction_factor::Int=3,
    bootstrap_count::Int=0)

Pruner using Hyperband.

Arguments

• min_resource::Int=1: A parameter for specifying the minimum resource allocated to a trial noted as r in the paper. A smaller r will give a result faster, but a larger r will give a better guarantee of successful judging between configurations.
• max_resource::Union{String,Int}="auto": A parameter for specifying the maximum resource allocated to a trial. This value represents and should match the maximum iteration steps (e.g., the number of epochs for neural networks). When this argument is "auto", the maximum resource is estimated according to the completed trials.
• reduction_factor::Int=3: A parameter for specifying reduction factor of promotable trials noted as η in the paper.
• bootstrap_count::Int=0: Parameter specifying the number of trials required in a rung before any trial can be promoted. Incompatible with max_resource="auto".

ThresholdPruner(;
    lower::Union{Float64,Nothing}=nothing,
    upper::Union{Float64,Nothing}=nothing,
    n_warmup_steps::Int=0,
    interval_steps::Int=1)

Pruner to detect outlying metrics of the trials. Prune if a metric exceeds upper threshold, falls behind lower threshold or reaches NaN.

Arguments

• lower::Union{Float64,Nothing}=nothing: A minimum value which determines whether pruner prunes or not. If an intermediate value is smaller than lower, it prunes.
• upper::Union{Float64,Nothing}=nothing: A maximum value which determines whether pruner prunes or not. If an intermediate value is larger than upper, it prunes.
• n_warmup_steps::Int=0: Pruning is disabled if the step is less than the given number of warmup steps.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported. Value must be at least 1.

WilcoxonPruner(;
    p_threshold::Float64=0.1,
    n_startup_steps::Int=2)

WilcoxonPruner depends on Scipy, which isnt shipped with this package by default. Pruner based on the Wilcoxon signed-rank test. This pruner performs the Wilcoxon signed-rank test between the current trial and the current best trial, and stops whenever the pruner is sure up to a given p-value that the current trial is worse than the best one.

Arguments

• p_threshold::Float64=0.1: The p-value threshold for pruning. This value should be between 0 and 1. A trial will be pruned whenever the pruner is sure up to the given p-value that the current trial is worse than the best trial. The larger this value is, the more aggressive pruning will be performed.
• n_startup_steps::Int=2: The number of steps before which no trials are pruned. Pruning starts only after you have nstartupsteps steps of available observations for comparison between the current trial and the best trial.

Study(study, artifact_stpre, storage)

This data structure represents an Optuna study and its corresponding artifact and data storage. A study is a collection of trials that share the same optimization objective.

load_study(
    storage::BaseStorage,
    study_name::String,
    artifact_store::BaseArtifactStore;
    sampler=nothing::Union{Nothing,BaseSampler},
    pruner=nothing::Union{Nothing,BasePruner},
)

Load an existing study with the given name, artifact store, storage, sampler and pruner.

Arguments

• study_name::String: Name of the study.
• artifact_store::BaseArtifactStore: Artifact store for the study. (see Artifacts)
• storage::BaseStorage: Storage of the study. (see Storage)

Keyword Arguments

• sampler::Union{Nothing,BaseSampler}=nothing: Sampler to use for the study. (see Sampler)
• pruner::Union{Nothing,BasePruner}=nothing: Pruner to use for the study. (see Pruner)

Returns

• Study: The loaded study. (see Study)

delete_study(
    study_name::String,
    storage::BaseStorage,
)

Delete a study with the given name and storage backend.

Arguments

• study_name::String: Name of the study.
• storage::BaseStorage: Storage of the study. (see Storage)

copy_study(
    from_study_name::String,
    from_storage::BaseStorage,
    to_storage::BaseStorage,
    to_study_name::String="",
)

Copy a study from one storage backend to another.

Arguments

• from_study_name::String: Name of the study to copy.
• from_storage::BaseStorage: Storage backend to copy the study from. (see Storage)
• to_storage::BaseStorage: Storage backend to copy the study to. (see Storage)
• to_study_name::String="": Name of the new study. If empty, the original study name is used.

ask(study::Study)

Wrapper for the Optuna ask function [ToDo: link URL]. This function is safe for multithreading.

Arguments

• study::Study: The study to ask the trial from. (see Study)

Keywords

• multithreading::Bool if multithreading is used, default is automatically detected (true if more than one thread is available)

Returns

• Trial: The new trial. (see Trial)

tell(study::Study, trial::Trial, score::Union{Nothing,T,Vector{T}}=nothing; prune::Bool=false

) where {T<:AbstractFloat}

Tell the study about the result of a trial. This is the proper way to complete a trial created with the ask function.

Arguments

• study::Study: The study to report the trial to. (see Study)
• trial::Trial: The trial which was completed. (see Trial)
• score::Union{Nothing,T,Vector{T}}=nothing: The score of the trial. If nothing, the trial is pruned.
• prune::Bool=false: If true, the trial is pruned.

best_trial(study::Study)

Get the best trial of the study.

Arguments

• study::Study: The study to get the best trial from. (see Study)

Returns

• Trial: The best trial. (see Trial)

best_params(study::Study)

Get the best parameters of the study.

Arguments

• study::Study: The study to get the best parameters from. (see Study)

Returns

• Dict{String,Any}: The best parameters.

best_value(study::Study)

Get the best objective value of the study.

Arguments

• study::Study: The study to get the best objective value from. (see Study)

Returns

• Float64: The best objective value.

upload_artifact(study::Study, trial::Trial, data::Dict)

Upload an artifact for a given trial in the study. The artifact is a .jld2 file containing the provided data.

Arguments

• study::Study: The study to upload the artifact to. (see Study)
• trial::Trial: The trial to associate the artifact with. (see Trial)
• data::Dict: The data to be stored as an artifact.

get_all_artifact_meta(study::Study)

Get all artifact metadata for all trials in the given study.

Arguments

• study::Study: The study to get the artifact metadata from. (see Study)

Returns

• Vector{ArtifactMeta}: List of artifact metadata of all artifacts in the study.

get_all_artifact_meta(study::Study, trial)

Get all artifact metadata for the trial in the given study.

Arguments

• study::Study: The study to get the artifact metadata from. (see Study)
• trial: The trial to get the artifact metadata from. (see Trial)

Returns

• Vector{ArtifactMeta}: List of artifact metadata of the given trial.

download_artifact(study::Study, artifact_id::String, file_path::String)

Download an artifact from a given study identified by its artifactid and the filepath where it should be stored.

Arguments

• study::Study: The study to download the artifact from. (see Study)
• artifact_id::String: The ID of the artifact to download.
• file_path::String: The path where the downloaded artifact should be stored.

Trial(trial)

Trial is a data structure wrapper for an Optuna trial.

suggest_int(trial::Trial, name::String, low::T, high::T) where {T<:Signed}

Suggest an integer value for the given parameter name within the specified range.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• low::T: The lower bound of the range (inclusive).
• high::T: The upper bound of the range (inclusive).

Returns

• T: Suggested integer value.

suggest_float(trial::Trial, name::String, low::T, high::T) where {T<:AbstractFloat}

Suggest a float value for the given parameter name within the specified range.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• low::T: The lower bound of the range (inclusive).
• high::T: The upper bound of the range (inclusive).

Returns

• T: Suggested float value.

suggest_categorical(trial::Trial, name::String, choices::Vector{T}) where {T<:Union{Bool,Int,AbstractFloat,String}}

Suggest a categorical value for the given parameter name from the specified choices.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• low::T: The lower bound of the range (inclusive).
• high::T: The upper bound of the range (inclusive).

Returns

• T: Suggested integer value.

report(trial::Trial, value::AbstractFloat, step::Int)

Report an intermediate value for the given trial at a specific step.

Arguments

• trial::Trial: The trial to report the value for. (see Trial)
• value::AbstractFloat: The intermediate value to report.
• step::Int: The step at which the value is reported.

should_prune(trial::Trial)

Check if the given trial should be pruned based on the pruner's decision.

Arguments

• trial::Trial: The trial to check for pruning. (see Trial)

Returns

• Bool: true if the trial should be pruned, false otherwise.

optimize(study::Study, objective::Function, params::NamedTuple; n_trials=100::Int)

Optimize the objective function by choosing a suitable set of hyperparameter values from the given params.

Uses the sampler of the study which implements the task of value suggestion based on a specified distribution. For the available samplers see Sampler.

If the objective function returns nothing, the trial is pruned.

Arguments

• study::Study: Study that should be optimized.
• objective::Function: Function that takes a trial and returns a score.
• params::NamedTuple: Named tuple of parameters to optimize.

Keyword Arguments

• n_trials::Int: Number of trials to run.

Returns

• Study: The optimized study. Study