API Reference

RDBStorage(
    url::String
)

Storage class for RDB backends. For further information see the RDBStorage in the Optuna python documentation.

Arguments

• url::String: URL of the database (e.g. directory and filename for SQLite).

InMemoryStorage()

Storage class that stores data in memory of the running process. For further information see the InMemoryStorage in the Optuna python documentation.

JournalStorage(
    backend::BaseJournalBackend
)

Storage class for Journal storage backend. For further information see the JournalStorage in the Optuna python documentation.

Arguments

• backend::BaseJournalBackend: Backend that determines where the data is stored.

get_all_study_names(
    storage::BaseStorage
)

Returns all study names stored in the given storage. For further information see the getallstudy_names in the Optuna python documentation.

Arguments

• storage::BaseStorage: Storage backend to query.

Returns

• Vector{String}: List of study names.

create_sqlite_url(
    database_path::String, 
    database_name::String
)

Returns a valid URL for a SQLite database with the given arguments.

Arguments

• database_path::String: Path to the directory where the database is stored.
• database_name::String: Name of the database file.

Returns

• String: URL for the SQLite database.

create_mysql_url(;
    user_name::String="",
    password::String="",
    host::String="",
    port::String="",
    database_name::String="",
    query::Dict{String,Any}=Dict{String,Any}(),

)

Returns a valid URL for a MySQL database with the given arguments.

Keyword Arguments

• user_name::String="": Name of the database user.
• password::String="": Password of the database user.
• host::String="": Host where the database server is running.
• port::String="": Port of the database server.
• database_name::String="": Name of the database.
• query::Dict{String,Any}=Dict{String,Any}(): Query string with keys and either strings or additional query strings as values.

Returns

• String: URL to the MySQL database.

create_redis_url(;
    user_name::String="",
    password::String="",
    host::String="",
    port::String="6379",
    database_index::String="0",

) Returns a valid URL for a Redis server with the given arguments.

Keyword Arguments

• user_name::String="": Name of the Redis user.
• password::String="": Password of the Redis user.
• host::String="": Host where the Redis server is running.
• port::String="6379": Port of the Redis server.
• database_index::String="0": Index of the database.

Returns

• String: URL to the Redis database.

JournalFileBackend(
    file_path::String; 
    lock_obj::Union{Nothing,BaseJournalFileLock}=nothing
)

File storage class for Journal log backend. For further information see the JournalFileBackend in the Optuna python documentation.

Arguments

• file_path::String: Path of file to persist the log to.

Keyword Arguments

• lock_obj::Union{Nothing,BaseJournalFileLock}=nothing: Lock object for process exclusivity. An instance of JournalFileSymlinkLock and JournalFileOpenLock can be passed.

JournalRedisBackend(
    url::String, 
    use_cluster::Bool=false, 
    prefix::String=""
)

Redis storage class for Journal log backend. For further information see the JournalRedisBackend in the Optuna python documentation.

Arguments

• url::String: URL of the redis storage.
• use_cluster::Bool=false: Flag whether you use the Redis cluster.
• prefix::String="": Prefix of the preserved key of logs. This is useful when multiple users work on one Redis server.

JournalFileSymlinkLock(
    file_path::String, 
    grace_period::Union{Nothing,Int}=30
)

Lock class for synchronizing processes for NFSv2 or later. For further information see the JournalFileSymlinkLock in the Optuna python documentation.

Arguments

• file_path::String: The path of the file whose race condition must be protected.
• grace_period::Union{Nothing,Int}=30: Grace period before an existing lock is forcibly released.

JournalFileOpenLock(
    file_path::String, 
    grace_period::Union{Nothing,Int}=30}
)

Lock class for synchronizing processes for NFSv3 or later. For further information see the JournalFileOpenLock in the Optuna python documentation.

Arguments

• file_path::String: The path of the file whose race condition must be protected.
• grace_period::Union{Nothing,Int}=30: Grace period before an existing lock is forcibly released.

FileSystemArtifactStore(
    path::String
)

Data structure for a file system based artifact store. For further information see the FileSystemArtifactStore in the Optuna python documentation.

Arguments

• path::String: Path to the directory where artifacts are stored.

ArtifactMeta

Data structure containing metadata for an artifact. For further information see the ArtifactMeta in the Optuna python documentation.

RandomSampler(seed=nothing::Union{Nothing,Integer})

An independent sampler that samples randomly. For further information see the RandomSampler in the Optuna python documentation.

Arguments

• seed::Union{Nothing,Integer}=nothing: Seed for the random number generator.

TPESampler(;
    consider_prior::Bool=true,
    prior_weight::Float64=1.0,
    consider_magic_clip::Bool=true,
    consider_endpoints::Bool=false,
    n_startup_trials::Integer=10,
    n_ei_candidates::Integer=24,
    gamma::Union{Nothing,Function}=nothing,
    weights::Union{Nothing,Function}=nothing,
    seed::Union{Nothing,Integer}=nothing,
    multivariate::Bool=false,
    group::Bool=false,
    warn_independent_sampling::Bool=true,
    constant_liar::Bool=false,
    constraints_func::Union{Nothing,Function}=nothing,
    categorical_distance_func::Union{Nothing,Function}=nothing,
)

Sampler using TPE (Tree-structured Parzen Estimator) algorithm. For further information see the TPESampler in the Optuna python documentation.

Keyword Arguments

• consider_prior::Bool=true: Enhance the stability of Parzen estimator by imposing a Gaussian prior when true. The prior is only effective if the sampling distribution is either FloatDistribution, or IntDistribution. Deprecated in v4.3.0. Will be removed in v6.0.0.
• prior_weight::Float64=1.0: The weight of the prior. This argument is used in FloatDistribution, IntDistribution, and CategoricalDistribution.
• consider_magic_clip::Bool=true: Enable a heuristic to limit the smallest variances of Gaussians used in the Parzen estimator.
• consider_endpoints::Bool=false: Take endpoints of domains into account when calculating variances of Gaussians in Parzen estimator. See the original paper for details on the heuristics to calculate the variances.
• n_startup_trials::Integer=10: The random sampling is used instead of the TPE algorithm until the given number of trials finish in the same study.
• n_ei_candidates::Integer=24: Number of candidate samples used to calculate the expected improvement.
• gamma::Union{Nothing,Function}=nothing: A function that takes the number of finished trials and returns the number of trials to form a density function for samples with low grains. See the original paper for more details.
• weights::Union{Nothing,Function}=nothing: A function that takes the number of finished trials and returns a weight for them. See Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures for more details.
• seed::Union{Nothing,Integer}=nothing: Seed for random number generator.
• multivariate::Bool=false: If this is true, the multivariate TPE is used when suggesting parameters. The multivariate TPE is reported to outperform the independent TPE. See BOHB: Robust and Efficient Hyperparameter Optimization at Scale and the article of the Optuna dev team for more details.
• group::Bool=false: If this and multivariate are true, the multivariate TPE with the group decomposed search space is used when suggesting parameters. The sampling algorithm decomposes the search space based on past trials and samples from the joint distribution in each decomposed subspace. The decomposed subspaces are a partition of the whole search space. Each subspace is a maximal subset of the whole search space, which satisfies the following: for a trial in completed trials, the intersection of the subspace and the search space of the trial becomes subspace itself or an empty set. Sampling from the joint distribution on the subspace is realized by multivariate TPE. If group is true, multivariate must be true as well.
• warn_independent_sampling::Bool=true: If this is true and multivariate=true, a warning message is emitted when the value of a parameter is sampled by using an independent sampler. If multivariate=false, this flag has no effect.
• constant_liar::Bool=false: If true, penalize running trials to avoid suggesting parameter configurations nearby.
• constraints_func::Union{Nothing,Function}=nothing: An optional function that computes the objective constraints. It must take a FrozenTrial and return the constraints. The return value must be a sequence of floats. A value strictly larger than 0 means that a constraints is violated. A value equal to or smaller than 0 is considered feasible. If constraintsfunc returns more than one value for a trial, that trial is considered feasible if and only if all values are equal to 0 or smaller. The constraintsfunc will be evaluated after each successful trial. The function won’t be called when trials fail or they are pruned, but this behavior is subject to change in the future releases.
• categorical_distance_func::Union{Nothing,Function}=nothing: A dictionary of distance functions for categorical parameters. The key is the name of the categorical parameter and the value is a distance function that takes two CategoricalChoiceTypes and returns a float value. The distance function must return a non-negative value. While categorical choices are handled equally by default, this option allows users to specify prior knowledge on the structure of categorical parameters. When specified, categorical choices closer to current best choices are more likely to be sampled.

GPSampler(;
    seed::Union{Nothing,Integer}=nothing,
    independent_sampler::Union{Nothing,BaseSampler}=nothing,
    n_startup_trials::Integer=10,
    deterministic_objective::Bool=false,
    constraints_func::Union{Nothing,Function}=nothing,
    warn_independent_sampling::Bool=true,
)

Sampler using Gaussian process-based Bayesian optimization. For further information see the GPSampler in the Optuna python documentation.

Keyword Arguments

• seed::Union{Nothing,Integer}=nothing: Random seed to initialize internal random number generator. Defaults to nothing (a seed is picked randomly).
• independent_sampler::Union{Nothing,BaseSampler}=nothing: Sampler used for initial sampling (for the first n_startup_trials trials) and for conditional parameters. Defaults to nothing (a random sampler with the same seed is used).
• n_startup_trials::Integer=10: Number of initial trials.
• deterministic_objective::Bool=false: Whether the objective function is deterministic or not. If true, the sampler will fix the noise variance of the surrogate model to the minimum value (slightly above 0 to ensure numerical stability). Defaults to false. Currently, all the objectives will be assume to be deterministic if true.
• constraints_func::Union{Nothing,Function}=nothing: An optional function that computes the objective constraints. It must take a FrozenTrial and return the constraints. The return value must be a sequence of floats. A value strictly larger than 0 means that a constraints is violated. A value equal to or smaller than 0 is considered feasible. If constraints_func returns more than one value for a trial, that trial is considered feasible if and only if all values are equal to 0 or smaller.
• warn_independent_sampling::Bool=true: If this is true, a warning message is emitted when the value of a parameter is sampled by using an independent sampler, meaning that no GP model is used in the sampling. Note that the parameters of the first trial in a study are always sampled via an independent sampler, so no warning messages are emitted in this case.

CmaEsSampler(
    x0::Union{Nothing,Dict{String,Any}}=nothing,
    sigma0::Union{Nothing,Float64}=nothing,
    n_startup_trials::Int=1,
    independent_sampler::Union{Nothing,BaseSampler}=nothing,
    warn_independent_sampling::Bool=true,
    seed::Union{Nothing,Integer}=nothing;
    consider_pruned_trials::Bool=false,
    restart_strategy::Union{Nothing,String}=nothing,
    popsize::Union{Nothing,Integer}=nothing,
    inc_popsize::Int=-1,
    use_separable_cma::Bool=false,
    with_margin::Bool=false,
    lr_adapt::Bool=false,
    source_trials::Union{Nothing,Vector{Trial}}=nothing,
)

A sampler using cmaes as the backend. For further information see the CmaEsSampler in the Optuna python documentation.

Arguments

• x0::Union{Nothing,Dict{String,Any}}=nothing: A dictionary of an initial parameter values for CMA-ES. By default, the mean of low and high for each distribution is used. Note that x0 is sampled uniformly within the search space domain for each restart if you specify restart_strategy argument.
• sigma0::Union{Nothing,Float64}=nothing: Initial standard deviation of CMA-ES. By default, sigma0 is set to minrange / 6, where minrange denotes the minimum range of the distributions in the search space.
• n_startup_trials::Int=1: The independent sampling is used instead of the CMA-ES algorithm until the given number of trials finish in the same study.
• independent_sampler::Union{Nothing,BaseSampler}=nothing: A BaseSampler instance that is used for independent sampling. The parameters not contained in the relative search space are sampled by this sampler. The search space for CmaEsSampler is determined by intersection_search_space(). If nothing is specified, RandomSampler is used as the default.
• warn_independent_sampling::Bool=true: If this is true, a warning message is emitted when the value of a parameter is sampled by using an independent sampler. Note that the parameters of the first trial in a study are always sampled via an independent sampler, so no warning messages are emitted in this case.
• seed::Union{Nothing,Integer}=nothing: A random seed for CMA-ES.

Keyword Arguments

• consider_pruned_trials::Bool=false: If this is true, the PRUNED trials are considered for sampling.
• restart_strategy::Union{Nothing,String}=nothing: Strategy for restarting CMA-ES optimization when converges to a local minimum. If nothing is given, CMA-ES will not restart (default). If ipop is given, CMA-ES will restart with increasing population size. if bipop is given, CMA-ES will restart with the population size increased or decreased. Please see also inc_popsize parameter. Deprecated in v4.4.0. Will be removed in v6.0.0.
• popsize::Union{Nothing,Integer}=nothing: A population size of CMA-ES.
• inc_popsize::Int=-1: Multiplier for increasing population size before each restart. This argument will be used when restart_strategy = 'ipop' or restart_strategy = 'bipop' is specified. Deprecated in v4.4.0. Will be removed in v6.0.0.
• use_separable_cma::Bool=false: If this is true, the covariance matrix is constrained to be diagonal. Due to reduce the model complexity, the learning rate for the covariance matrix is increased. Consequently, this algorithm outperforms CMA-ES on separable functions.
• with_margin::Bool=false: If this is true, CMA-ES with margin is used. This algorithm prevents samples in each discrete distribution (FloatDistribution with step and IntDistribution) from being fixed to a single point. Currently, this option cannot be used with use_separable_cma=true.
• lr_adapt::Bool=false: If this is true, CMA-ES with learning rate adaptation is used. This algorithm focuses on working well on multimodal and/or noisy problems with default settings. Currently, this option cannot be used with use_separable_cma=true or with_margin=true.
• source_trials::Union{Nothing,Vector{Trial}}=nothing: This option is for Warm Starting CMA-ES, a method to transfer prior knowledge on similar HPO tasks through the initialization of CMA-ES. This method estimates a promising distribution from sourcetrials and generates the parameter of multivariate gaussian distribution. Please note that it is prohibited to use x0, sigma0, or `useseparable_cma` argument together.

NSGAIISampler(;
    population_size::Int=50,
    mutation_prob::Union{Nothing,Float64}=nothing,
    crossover::Union{Nothing,BaseCrossover}=nothing,
    crossover_prob::Float64=0.9,
    swapping_prob::Float64=0.5,
    seed::Union{Nothing,Integer}=nothing,
    constraints_func::Union{Nothing,Function}=nothing,
    elite_population_selection_strategy::Union{Nothing,Function}=nothing,
    child_generation_strategy::Union{Nothing,Function}=nothing,
    after_trial_strategy::Union{Nothing,Function}=nothing,
)

Multi-objective sampler using the NSGA-II algorithm. For further information see the NSGAIISampler in the Optuna python documentation.

Keyword Arguments

• population_size::Int=50: Number of individuals (trials) in a generation. population_size must be greater than or equal to crossover.n_parents. For UNDXCrossover and SPXCrossover, n_parents=3, and for the other algorithms, n_parents=2.
• mutation_prob::Union{Nothing,Float64}=nothing: Probability of mutating each parameter when creating a new individual. If nothing is specified, the value 1.0 / len(parenttrial.params) is used where `parenttrial` is the parent trial of the target individual.
• crossover::Union{Nothing,BaseCrossover}=nothing: Crossover to be applied when creating child individuals. The available crossovers are listed here. UniformCrossover is always applied to parameters sampled from CategoricalDistribution, and by default for parameters sampled from other distributions unless this argument is specified. For more information on each of the crossover method, please refer to specific crossover documentation.
• crossover_prob::Float64=0.9: Probability that a crossover (parameters swapping between parents) will occur when creating a new individual.
• swapping_prob::Float64=0.5: Probability of swapping each parameter of the parents during crossover.
• seed::Union{Nothing,Integer}=nothing: Seed for random number generator.
• constraints_func::Union{Nothing,Function}=nothing: An optional function that computes the objective constraints. It must take a FrozenTrial and return the constraints. The return value must be a sequence of floats. A value strictly larger than 0 means that a constraints is violated. A value equal to or smaller than 0 is considered feasible. If constraints_func returns more than one value for a trial, that trial is considered feasible if and only if all values are equal to 0 or smaller. The constraints_func will be evaluated after each successful trial. The function won’t be called when trials fail or they are pruned, but this behavior is subject to change in the future releases. The constraints are handled by the constrained domination. A trial x is said to constrained-dominate a trial y, if any of the following conditions is true: Trial x is feasible and trial y is not. Trial x and y are both infeasible, but trial x has a smaller overall violation. Trial x and y are feasible and trial x dominates trial y.
• elite_population_selection_strategy::Union{Nothing,Function}=nothing: The selection strategy for determining the individuals to survive from the current population pool. Default to nothing.
• child_generation_strategy::Union{Nothing,Function}=nothing: The strategy for generating child parameters from parent trials. Defaults to nothing.
• after_trial_strategy::Union{Nothing,Function}=nothing: A set of procedure to be conducted after each trial. Defaults to nothing.

NSGAIIISampler(;
    population_size::Int=50,
    mutation_prob::Union{Nothing,Float64}=nothing,
    crossover::Union{Nothing,BaseCrossover}=nothing,
    crossover_prob::Float64=0.9,
    swapping_prob::Float64=0.5,
    seed::Union{Nothing,Integer}=nothing,
    constraints_func::Union{Nothing,Function}=nothing,
    reference_points::Union{Nothing,AbstractArray{T,2}}=nothing,
    dividing_parameter::Int=3,
    elite_population_selection_strategy::Union{Nothing,Function}=nothing,
    child_generation_strategy::Union{Nothing,Function}=nothing,
    after_trial_strategy::Union{Nothing,Function}=nothing,
) where {T}

Multi-objective sampler using the NSGA-III algorithm. For further information see the NSGAIIISampler in the Optuna python documentation.

Keyword Arguments

• population_size::Int=50: Number of individuals (trials) in a generation. population_size must be greater than or equal to crossover.n_parents. For UNDXCrossover and SPXCrossover, n_parents=3, and for the other algorithms, n_parents=2.
• mutation_prob::Union{Nothing,Float64}=nothing: Probability of mutating each parameter when creating a new individual. If nothing is specified, the value 1.0 / len(parenttrial.params) is used where `parenttrial` is the parent trial of the target individual.
• crossover::Union{Nothing,BaseCrossover}=nothing: Crossover to be applied when creating child individuals. The available crossovers are listed here. UniformCrossover is always applied to parameters sampled from CategoricalDistribution, and by default for parameters sampled from other distributions unless this argument is specified. For more information on each of the crossover method, please refer to specific crossover documentation.
• crossover_prob::Float64=0.9: Probability that a crossover (parameters swapping between parents) will occur when creating a new individual.
• swapping_prob::Float64=0.5: Probability of swapping each parameter of the parents during crossover.
• seed::Union{Nothing,Integer}=nothing: Seed for random number generator.
• constraints_func::Union{Nothing,Function}=nothing: An optional function that computes the objective constraints. It must take a FrozenTrial and return the constraints. The return value must be a sequence of floats. A value strictly larger than 0 means that a constraints is violated. A value equal to or smaller than 0 is considered feasible. If constraints_func returns more than one value for a trial, that trial is considered feasible if and only if all values are equal to 0 or smaller. The constraints_func will be evaluated after each successful trial. The function won’t be called when trials fail or they are pruned, but this behavior is subject to change in the future releases. The constraints are handled by the constrained domination. A trial x is said to constrained-dominate a trial y, if any of the following conditions is true: Trial x is feasible and trial y is not. Trial x and y are both infeasible, but trial x has a smaller overall violation. Trial x and y are feasible and trial x dominates trial y.
• reference_points::Union{Nothing,AbstractArray{T,2}}=nothing: A 2 dimension array with objective dimension columns. Represents a list of reference points which is used to determine who to survive. After non-dominated sort, who out of borderline front are going to survived is determined according to how sparse the closest reference point of each individual is. In the default setting the algorithm uses uniformly spread points to diversify the result. It is also possible to reflect your preferences by giving an arbitrary set of target points since the algorithm prioritizes individuals around reference points.
• dividing_parameter::Int=3: A parameter to determine the density of default reference points. This parameter determines how many divisions are made between reference points on each axis. The smaller this value is, the less reference points you have. The default value is 3. Note that this parameter is not used when reference_points is not nothing.
• elite_population_selection_strategy::Union{Nothing,Function}=nothing: The selection strategy for determining the individuals to survive from the current population pool. Default to nothing.
• child_generation_strategy::Union{Nothing,Function}=nothing: The strategy for generating child parameters from parent trials. Defaults to nothing.
• after_trial_strategy::Union{Nothing,Function}=nothing: A set of procedure to be conducted after each trial. Defaults to nothing.

GridSampler(
    search_space::Dict{String,Vector},
    seed::Union{Nothing,Integer}=nothing
)

Sampler using grid search. For further information see the GridSampler in the Optuna python documentation.

Arguments

• search_space::Dict{String, Vector}: A dictionary whose key and value are a parameter name and the corresponding candidates of values, respectively.
• seed::Union{Nothing,Integer}=nothing: A seed to fix the order of trials as the grid is randomly shuffled. This shuffle is beneficial when the number of grids is larger than n_trials in optimize() to suppress suggesting similar grids. Please note that fixing seed for each process is strongly recommended in distributed optimization to avoid duplicated suggestions.

QMCSampler(;
    qmc_type::String="sobol",
    scramble::Bool=false,
    seed::Union{Nothing,Integer}=nothing,
    independent_sampler::Union{Nothing,BaseSampler}=nothing,
    warn_asynchronous_seeding::Bool=true,
    warn_independent_sampling::Bool=true,
)

A Quasi Monte Carlo Sampler that generates low-discrepancy sequences. For further information see the QMCSampler in the Optuna python documentation.

Keyword Arguments

• qmc_type::String="sobol": The type of QMC sequence to be sampled. This must be one of “halton” and “sobol”. Default is “sobol”.
• scramble::Bool=false: If this option is true, scrambling (randomization) is applied to the QMC sequences.
• seed::Union{Nothing,Integer}: A seed for QMCSampler. This argument is used only when scramble is true. If this is nothing, the seed is initialized randomly. Default is nothing.
• independent_sampler::Union{Nothing,BaseSampler}=nothing: A BaseSampler instance that is used for independent sampling. The first trial of the study and the parameters not contained in the relative search space are sampled by this sampler. If nothing is specified, RandomSampler is used as the default.
• warn_asynchronous_seeding::Bool=true: If this is true, a warning message is emitted when the scrambling (randomization) is applied to the QMC sequence and the random seed of the sampler is not set manually.
• warn_independent_sampling::Bool=true: If this is true, a warning message is emitted when the value of a parameter is sampled by using an independent sampler. Note that the parameters of the first trial in a study are sampled via an independent sampler in most cases, so no warning messages are emitted in such cases.

BruteForceSampler(
    seed::Union{Nothing,Integer}=nothing,
    avoid_premature_stop::Bool=false
)

Sampler using brute force. This sampler performs exhaustive search on the defined search space. For further information see the BruteForceSampler in the Optuna python documentation.

Arguments

• seed::Union{Nothing,Integer}=nothing: A seed to fix the order of trials as the search order randomly shuffled. Please note that it is not recommended using this option in distributed optimization settings since this option cannot ensure the order of trials and may increase the number of duplicate suggestions during distributed optimization.
• avoid_premature_stop::Bool=false:If true, the sampler performs a strict exhaustive search. Please note that enabling this option may increase the likelihood of duplicate sampling. When this option is not enabled (default), the sampler applies a looser criterion for determining when to stop the search, which may result in incomplete coverage of the search space. For more information, see https://github.com/optuna/optuna/issues/5780.

PartialFixedSampler(
    fixed_params::Dict{String,Any},
    base_sampler::BaseSampler)

Sampler with partially fixed parameters. For further information see the PartialFixedSampler in the Optuna python documentation.

Arguments

• fixed_params::Dict{String,Any}: A dictionary of parameters to be fixed.
• base_sampler::BaseSampler: A sampler which samples unfixed parameters.

MedianPruner(
    n_startup_trials::Int=5,
    n_warmup_steps::Int=0,
    interval_steps::Int=1;
    n_min_trials::Int=1)

Pruner using the median stopping rule. Prune if the trial's best intermediate result is worse than median of intermediate results of previous trials at the same step. For further information see the MedianPruner in the Optuna python documentation.

Arguments

• n_startup_trials::Int=5: Pruning is disabled until the given number of trials finish in the same study.
• n_warmup_steps::Int=0: Pruning is disabled until the trial exceeds the given number of step. Note that this feature assumes that step starts at zero.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported.
• n_min_trials::Int=1: Minimum number of reported trial results at a step to judge whether to prune. If the number of reported intermediate values from all trials at the current step is less than n_min_trials, the trial will not be pruned.

NopPruner()

Pruner which never prunes trials. For further information see the NopPruner in the Optuna python documentation.

PatientPruner(
    wrapped_pruner::Union{BasePruner,Nothing},
    patience::Int;
    min_delta::Float64=0.0)

Pruner which wraps another pruner with tolerance. This pruner monitors intermediate values in a trial and prunes the trial if the improvement in the intermediate values after a patience period is less than a threshold. For further information see the PatientPruner in the Optuna python documentation.

Arguments

• wrapped_pruner::Union{BasePruner,Nothing}: Wrapped pruner to perform pruning when PatientPruner allows a trial to be pruned. If it is nothing, this pruner is equivalent to early-stopping taken the intermediate values in the individual trial.
• patience::Int: Pruning is disabled until the objective doesn't improve for patience consecutive steps.
• min_delta::Float64=0.0: Tolerance value to check whether or not the objective improves. This value should be non-negative.

PercentilePruner(
    percentile::Float64,
    n_startup_trials::Int=5,
    n_warmup_steps::Int=0,
    interval_steps::Int=1;
    n_min_trials::Int=1)

Pruner to keep the specified percentile of the trials. Prune if the best intermediate value is in the bottom percentile among trials at the same step. For further information see the PercentilePruner in the Optuna python documentation.

Arguments

• percentile::Float64: Percentile which must be between 0 and 100 inclusive (e.g., When given 25.0, top of 25th percentile trials are kept).
• n_startup_trials::Int=5: Pruning is disabled until the given number of trials finish in the same study.
• n_warmup_steps::Int=0: Pruning is disabled until the trial exceeds the given number of step. Note that this feature assumes that step starts at zero.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported. Value must be at least 1.
• n_min_trials::Int=1: Minimum number of reported trial results at a step to judge whether to prune. If the number of reported intermediate values from all trials at the current step is less than n_min_trials, the trial will not be pruned.

SuccessiveHalvingPruner(;
    min_resource::Union{String,Int}="auto",
    reduction_factor::Int=4,
    min_early_stopping_rate::Int=0,
    bootstrap_count::Int=0)

Pruner using Asynchronous Successive Halving Algorithm. For further information see the SuccessiveHalvingPruner in the Optuna python documentation.

Keyword Arguments

• min_resource::Union{String,Int}="auto": A parameter for specifying the minimum resource allocated to a trial (in the paper this parameter is referred to as r). This parameter defaults to "auto" where the value is determined based on a heuristic that looks at the number of required steps for the first trial to complete.
• reduction_factor::Int=4: A parameter for specifying reduction factor of promotable trials (in the paper this parameter is referred to as η). At the completion point of each rung, about 1/reduction_factor trials will be promoted.
• min_early_stopping_rate::Int=0: A parameter for specifying the minimum early-stopping rate (in the paper this parameter is referred to as s).
• bootstrap_count::Int=0: Minimum number of trials that need to complete a rung before any trial is considered for promotion into the next rung.

HyperbandPruner(;
    min_resource::Int=1,
    max_resource::Union{String,Int}="auto",
    reduction_factor::Int=3,
    bootstrap_count::Int=0)

Pruner using Hyperband. For further information see the HyperbandPruner in the Optuna python documentation.

Keyword Arguments

• min_resource::Int=1: A parameter for specifying the minimum resource allocated to a trial noted as r in the paper. A smaller r will give a result faster, but a larger r will give a better guarantee of successful judging between configurations.
• max_resource::Union{String,Int}="auto": A parameter for specifying the maximum resource allocated to a trial. This value represents and should match the maximum iteration steps (e.g., the number of epochs for neural networks). When this argument is "auto", the maximum resource is estimated according to the completed trials.
• reduction_factor::Int=3: A parameter for specifying reduction factor of promotable trials noted as η in the paper.
• bootstrap_count::Int=0: Parameter specifying the number of trials required in a rung before any trial can be promoted. Incompatible with max_resource="auto".

ThresholdPruner(;
    lower::Union{Float64,Nothing}=nothing,
    upper::Union{Float64,Nothing}=nothing,
    n_warmup_steps::Int=0,
    interval_steps::Int=1)

Pruner to detect outlying metrics of the trials. Prune if a metric exceeds upper threshold, falls behind lower threshold or reaches NaN. For further information see the ThresholdPruner in the Optuna python documentation.

Keyword Arguments

• lower::Union{Float64,Nothing}=nothing: A minimum value which determines whether pruner prunes or not. If an intermediate value is smaller than lower, it prunes.
• upper::Union{Float64,Nothing}=nothing: A maximum value which determines whether pruner prunes or not. If an intermediate value is larger than upper, it prunes.
• n_warmup_steps::Int=0: Pruning is disabled if the step is less than the given number of warmup steps.
• interval_steps::Int=1: Interval in number of steps between the pruning checks, offset by the warmup steps. If no value has been reported at the time of a pruning check, that particular check will be postponed until a value is reported. Value must be at least 1.

WilcoxonPruner(;
    p_threshold::Float64=0.1,
    n_startup_steps::Int=2)

Pruner based on the Wilcoxon signed-rank test. This pruner performs the Wilcoxon signed-rank test between the current trial and the current best trial, and stops whenever the pruner is sure up to a given p-value that the current trial is worse than the best one. For further information see the WilcoxonPruner in the Optuna python documentation.

Keyword Arguments

• p_threshold::Float64=0.1: The p-value threshold for pruning. This value should be between 0 and 1. A trial will be pruned whenever the pruner is sure up to the given p-value that the current trial is worse than the best trial. The larger this value is, the more aggressive pruning will be performed.
• n_startup_steps::Int=2: The number of steps before which no trials are pruned. Pruning starts only after you have nstartupsteps steps of available observations for comparison between the current trial and the best trial.

UniformCrossover(
    swapping_prob::Float64=0.5
)

Select each parameter with equal probability from the two parent individuals. For further information see the UniformCrossover in the Optuna python documentation.

Arguments

• swapping_prob::Float64=0.5: Probability of swapping each parameter of the parents during crossover.

BLXAlphaCrossover(
    alpha::Float64=0.5
)

Uniformly samples child individuals from the hyper-rectangles created by the two parent individuals. For further information see the BLXAlphaCrossover in the Optuna python documentation.

Arguments

• alpha::Float64=0.5: Parametrizes blend operation.

SPXCrossover(
    epsilon::Union{Nothing,Float64}=nothing
)

Uniformly samples child individuals from within a single simplex that is similar to the simplex produced by the parent individual. For further information see the SPXCrossover in the Optuna python documentation.

Arguments

• epsilon::Union{Nothing,Float64}=nothing: Expansion rate. If not specified, defaults to sqrt(len(search_space) + 2).

SBXCrossover(
    eta::Union{Nothing,Float64}=nothing,
    uniform_crossover_prob::Float64=0.5,
    use_child_gene_prob::Float64=0.5,
)

Uniformly samples child individuals from within a single simplex that is similar to the simplex produced by the parent individual. For further information see the SBXCrossover in the Optuna python documentation.

Arguments

• eta::Union{Nothing,Float64}=nothing: Distribution index. A small value of eta allows distant solutions to be selected as children solutions. If not specified, takes default value of 2 for single objective functions and 20 for multi objective.
• uniform_crossover_prob::Float64=0.5: uniform_crossover_prob is the probability of uniform crossover between two individuals selected as candidate child individuals. This argument is whether or not two individuals are crossover to make one child individual. If the uniform_crossover_prob exceeds 0.5, the result is equivalent to 1-uniform_crossover_prob, because it returns one of the two individuals of the crossover result. If not specified, takes default value of 0.5. The range of values is [0.0, 1.0].
• use_child_gene_prob::Float64=0.5: use_child_gene_prob is the probability of using the value of the generated child variable rather than the value of the parent. This probability is applied to each variable individually. where 1-use_child_gene_prob is the probability of using the parent’s values as it is. If not specified, takes default value of 0.5. The range of values is (0.0, 1.0].

VSBXCrossover(
    eta::Union{Nothing,Float64}=nothing,
    uniform_crossover_prob::Float64=0.5,
    use_child_gene_prob::Float64=0.5,
)

vSBX generates child individuals without excluding any region of the parameter space, while maintaining the excellent properties of SBX. For further information see the VSBXCrossover in the Optuna python documentation.

Arguments

• eta::Union{Nothing,Float64}=nothing: Distribution index. A small value of eta allows distant solutions to be selected as children solutions. If not specified, takes default value of 2 for single objective functions and 20 for multi objective.
• uniform_crossover_prob::Float64=0.5: uniform_crossover_prob is the probability of uniform crossover between two individuals selected as candidate child individuals. This argument is whether or not two individuals are crossover to make one child individual. If the uniform_crossover_prob exceeds 0.5, the result is equivalent to 1-uniform_crossover_prob, because it returns one of the two individuals of the crossover result. If not specified, takes default value of 0.5. The range of values is [0.0, 1.0].
• use_child_gene_prob::Float64=0.5: use_child_gene_prob is the probability of using the value of the generated child variable rather than the value of the parent. This probability is applied to each variable individually. where 1-use_child_gene_prob is the probability of using the parent’s values as it is. If not specified, takes default value of 0.5. The range of values is (0.0, 1.0].

UNDXCrossover(
    sigma_xi::Float64=0.5, 
    sigma_eta::Union{Nothing,Float64}=nothing
)

Generates child individuals from the three parents using a multivariate normal distribution. For further information see the UNDXCrossover in the Optuna python documentation.

Arguments

• sigma_xi::Float64=0.5: Parametrizes normal distribution from which xi is drawn.
• sigma_eta::Union{Nothing,Float64}=nothing: Parametrizes normal distribution from which etas are drawn. If not specified, defaults to 0.35 / sqrt(len(search_space)).

Study(
    study, 
    artifact_store, 
    storage
)

This data structure represents an Optuna study and its corresponding artifact and data storage. A study is a collection of trials that share the same optimization objective.

load_study(
    study_name::String,
    storage::BaseStorage,
    artifact_store::BaseArtifactStore;
    sampler::Union{Nothing,BaseSampler}=nothing,
    pruner::Union{Nothing,BasePruner}=nothing,
)

Load an existing study with the given name, artifact store, storage, sampler and pruner. For further information see the load_study in the Optuna python documentation.

Arguments

• study_name::String: Name of the study.
• storage::BaseStorage: Storage of the study. (see Storage)
• artifact_store::BaseArtifactStore: Artifact store for the study. (see Artifacts)

Keyword Arguments

• sampler::Union{Nothing,BaseSampler}=nothing: Sampler to use for the study. (see Sampler)
• pruner::Union{Nothing,BasePruner}=nothing: Pruner to use for the study. (see Pruner)

Returns

• Study: The loaded study. (see Study)

delete_study(
    study_name::String,
    storage::BaseStorage,
)

Delete a study with the given name and storage backend. For further information see the delete_study in the Optuna python documentation.

Arguments

• study_name::String: Name of the study.
• storage::BaseStorage: Storage of the study. (see Storage)

copy_study(
    from_study_name::String,
    from_storage::BaseStorage,
    to_storage::BaseStorage,
    to_study_name::String="",
)

Copy a study from one storage backend to another. For further information see the copy_study in the Optuna python documentation.

Arguments

• from_study_name::String: Name of the study to copy.
• from_storage::BaseStorage: Storage backend to copy the study from. (see Storage)
• to_storage::BaseStorage: Storage backend to copy the study to. (see Storage)
• to_study_name::String="": Name of the new study. If empty, the original study name is used.

ask(
    study::Study; 
    multithreading::Bool=Threads.nthreads() > 1
)

Wrapper for the Optuna ask function. For further information see the ask in the Optuna python documentation.

This function is safe for multithreading.

Arguments

• study::Study: The study to ask the trial from. (see Study)

Keywords

• multithreading::Bool=Threads.nthreads() > 1 if multithreading is used, default is automatically detected (true if more than one thread is available)

Returns

• Trial: The new trial. (see Trial)

tell(
    study::Study,
    trial::Trial{false},
    score::Union{Nothing,T,Vector{T}}=nothing;
    prune::Bool=false,

) where {T<:AbstractFloat}

Tell the study about the result of a trial. This is the proper way to complete a trial created with the ask function. For further information see the tell in the Optuna python documentation.

Arguments

• study::Study: The study to report the trial to. (see Study)
• trial::Trial: The trial which was completed. (see Trial)
• score::Union{Nothing,T,Vector{T}}=nothing: The score of the trial. If nothing, the trial is pruned.
• prune::Bool=false: If true, the trial is pruned.

best_trial(
    study::Study
)

Get the best trial of the study. For further information see the best_trial in the Optuna python documentation.

Arguments

• study::Study: The study to get the best trial from. (see Study)

Returns

• Trial: The best trial. (see Trial)

best_params(
    study::Study
)

Get the best parameters of the study. For further information see the best_params in the Optuna python documentation.

Arguments

• study::Study: The study to get the best parameters from. (see Study)

Returns

• Dict{String,Any}: The best parameters.

best_value(
    study::Study
)

Get the best objective value of the study. For further information see the best_value in the Optuna python documentation.

Arguments

• study::Study: The study to get the best objective value from. (see Study)

Returns

• Float64: The best objective value.

upload_artifact(
    study::Study, 
    trial::Trial, 
    data::Dict
)

Upload an artifact for a given trial in the study. The artifact is a .jld2 file containing the provided data. For further information see the upload_artifact in the Optuna python documentation.

Arguments

• study::Study: The study to upload the artifact to. (see Study)
• trial::Trial: The trial to associate the artifact with. (see Trial)
• data::Dict: The data to be stored as an artifact.

get_all_artifact_meta(
    study::Study
)

Get all artifact metadata for all trials in the given study. For further information see the getallartifact_meta in the Optuna python documentation.

Arguments

• study::Study: The study to get the artifact metadata from. (see Study)

Returns

• Vector{ArtifactMeta}: List of artifact metadata of all artifacts in the study.

get_all_artifact_meta(
    study::Study, 
    trial
)

Get all artifact metadata for the trial in the given study. For further information see the getallartifact_meta in the Optuna python documentation.

Arguments

• study::Study: The study to get the artifact metadata from. (see Study)
• trial: The trial to get the artifact metadata from. (see Trial)

Returns

• Vector{ArtifactMeta}: List of artifact metadata of the given trial.

download_artifact(
    study::Study, 
    artifact_id::String, 
    file_path::String
)

Download an artifact from a given study identified by its artifactid and the filepath where it should be stored. For further information see the download_artifact in the Optuna python documentation.

Arguments

• study::Study: The study to download the artifact from. (see Study)
• artifact_id::String: The ID of the artifact to download.
• file_path::String: The path where the downloaded artifact should be stored.

Trial(
    trial, 
    multithreading::Bool=false
)

Trial is a data structure wrapper for an Optuna trial.

suggest_int(
    trial::Trial,
    name::String,
    low::T,
    high::T;
    step::T=1,
    log::Bool=false
) where {T<:Signed}

Suggest an integer value for the given parameter name within the specified range. For further information see the suggest_int in the Optuna python documentation.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• low::T: The lower bound of the range (inclusive).
• high::T: The upper bound of the range (inclusive).

Keyword Arguments

• step::T=1: The step size for the range. The suggested value will be a multiple of step away from low.
• log::Bool=false: If true, the range will be sampled on a logarithmic scale. (

Returns

• T: Suggested integer value.

suggest_float(
    trial::Trial,
    name::String,
    low::T,
    high::T;
    step::Union{Nothing,T}=nothing,
    log::Bool=false,
) where {T<:AbstractFloat}

Suggest a float value for the given parameter name within the specified range. For further information see the suggest_float in the Optuna python documentation.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• low::T: The lower bound of the range (inclusive).
• high::T: The upper bound of the range (inclusive).

Keyword Arguments

• step::Union{Nothing,T}=nothing: A step of discretization.
• log::Bool=false: If true, the range will be sampled on a logarithmic scale. (

Returns

• Float64: Suggested float value.

suggest_categorical(
    trial::Trial,
    name::String,
    choices::Union{Vector{T},Tuple{Vararg{T}}}
) where {T<:Union{Bool,Int,AbstractFloat,String}}

Suggest a categorical value for the given parameter name from the specified choices. For further information see the suggest_categorical in the Optuna python documentation.

Arguments

• trial::Trial: The trial to suggest the parameter for. (see Trial)
• name::String: The name of the parameter to suggest.
• choices::Union{Vector{T},Tuple{Vararg{T}}}: The choices to suggest from.

Returns

• T: Suggested categorical value.

report(
    trial::Trial,
    value::AbstractFloat,
    step::Int
)

Report an intermediate value for the given trial at a specific step. For further information see the report in the Optuna python documentation.

Arguments

• trial::Trial: The trial to report the value for. (see Trial)
• value::AbstractFloat: The intermediate value to report.
• step::Int: The step at which the value is reported.

should_prune(
    trial::Trial
)

Check if the given trial should be pruned based on the pruner's decision. For further information see the should_prune in the Optuna python documentation.

Arguments

• trial::Trial: The trial to check for pruning. (see Trial)

Returns

• Bool: true if the trial should be pruned, false otherwise.

optimize(
    study::Study,
    objective::Function,
    params::Union{Nothing,NamedTuple}=nothing;
    n_trials::Int=100,
    verbose::Bool=false,
    n_jobs::Integer=1,
)

Optimize function that checks if the optimization should be performed in a single-threaded or multi-threaded way based on the keyword argument n_jobs and calls the corresponding function.

Uses the sampler of the study which implements the task of value suggestion based on a specified distribution. For the available samplers see Sampler.

If the objective function returns nothing, the trial is pruned.

Arguments

• study::Study: Study that should be optimized.
• objective::Function: Function that takes a trial and returns a score.
• params::Union{Nothing,NamedTuple}=nothing: Named tuple of parameters to optimize. If nothing, the objective function is expected to handle the parameter suggestion internally. If a NamedTuple is passed, the objective function is expected to take the parameters as keyword arguments or as a NamedTuple (see examples).

Keyword Arguments

• n_trials::Int=100: Number of trials to run.
• verbose::Bool=false: If true, print information about the optimization process.
• n_jobs::Integer=1: Number of threads to use for optimization. If n_jobs > 1, multithreading is used. Note that the number of threads allocated for the process must be greater than or equal to n_jobs and exactly one interactive thread must be allocated for multithreading to work. You can start Julia with n threads and an interactive thread by setting the environment variable JULIA_NUM_THREADS=n,1 or start the Julia REPL with -t n,1.

Returns

• Study: The optimized study. Study

is_conda_pkg_installed(
    pkg_name::String; 
    version::Union{Nothing,String}=nothing
)

Test if the given package with the given version is installed in the conda environment.

Arguments

• pkg_name::String: Name of the package.

Keyword Arguments

• version::Union{Nothing,String}=nothing: Version of the package.

add_conda_pkg(
    pkg_name::String; 
    version::Union{Nothing,String}=nothing
)

Adds the given package with the given version in the conda environment if it is not installed.

Arguments

• pkg_name::String: Name of the package.

Keyword Arguments

• version::Union{Nothing,String}=nothing: Version of the package.