ANN models

ecopann.element

class ecopann.element.ELU_1

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class ecopann.element.Sigmoid_1

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class ecopann.element.Softplus_1

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class ecopann.element.Softplus_2

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

ecopann.element.activation(activation_name='rrelu')

Activation functions.

Parameters

activation_name (str, optional) – The name of activation function, which can be ‘relu’, ‘leakyrelu’, ‘prelu’, ‘rrelu’, ‘relu6’, ‘elu’, ‘celu’, ‘selu’, ‘silu’, ‘sigmoid’, ‘logsigmoid’, ‘tanh’, ‘tanhshrink’, ‘softsign’, or ‘softplus’. Default: ‘rrelu’

Returns

Activation functions.

Return type

object

Note

Although many activation functions are available, the recommended activation function is ‘rrelu’.

ecopann.element.avgPool1d(kernel_size)

ecopann.element.avgPool2d(kernel_size)

ecopann.element.avgPool3d(kernel_size)

ecopann.element.celu()

ecopann.element.dropout()

ecopann.element.dropout2d()

ecopann.element.dropout3d()

ecopann.element.elu()

ecopann.element.elu_1()

ecopann.element.get_dropout(drouput_name='dropout')

Get the dropout.

ecopann.element.leakyrelu()

ecopann.element.logsigmoid()

ecopann.element.maxPool1d(kernel_size)

ecopann.element.maxPool2d(kernel_size)

ecopann.element.maxPool3d(kernel_size)

ecopann.element.pooling(pool_name='maxPool2d', kernel_size=2)

ecopann.element.prelu()

ecopann.element.relu()

ecopann.element.relu6()

ecopann.element.rrelu()

ecopann.element.selu()

ecopann.element.sigmoid()

ecopann.element.sigmoid_1()

ecopann.element.silu()

ecopann.element.softplus()

ecopann.element.softplus_1()

ecopann.element.softplus_2()

ecopann.element.softsign()

ecopann.element.tanh()

ecopann.element.tanhshrink()

ecopann.nodeframe

ecopann.nodeframe.decreasingNode(node_in=1970, node_out=5, hidden_layer=3, get_allNode=True)

A network structure that the number of neurons in each hidden layer is decreased proportionally.

Parameters

• node_in (int) – The number of nodes in the input layer.

• node_out (int) – The number of nodes in the output layer.

• hidden_layer (int) – The number of the hidden layers.

• get_allNode (bool) – If True, return the number of all nodes, otherwise, only return the number of nodes of hidden layers. Default: True

Returns

A list that contains the number of nodes in each layer.

Return type

list

ecopann.sequence

class ecopann.sequence.Activation

Bases: object

Activation functions, to be used by class LinearSeq.

class ecopann.sequence.BatchNorm

Bases: object

Batch Normalization, to be used by class LinearSeq.

class ecopann.sequence.Dropout

Bases: object

Dropout, to be used by class LinearSeq.

class ecopann.sequence.LinearSeq(nodes, mainBN=True, finalBN=False, mainActive='rrelu', finalActive='None', mainDropout='None', finalDropout='None')

Bases: SeqName, BatchNorm, Activation, Dropout

Sequence of Linear.

Parameters

• nodes (list) – A list that contains the number of nodes in each layer.

• mainBN (bool, optional) – If True, the network will contain batch normalization layer in its main part. Default: True

• finalBN (bool, optional) – If True, the network will contain batch normalization layer in its last layer. Default: False

• mainActive (str, optional) – The activation function used in the main part of the network. See activation() for the available activation functions. Default: ‘rrelu’

• finalActive (str, optional) – The activation function used in the last layer of the network. See activation() for the available activation functions. Default: ‘None’

• mainDropout (str, optional) – The dropout used in the main part of the network, ‘None’ or ‘dropout’. Default: ‘None’

• finalDropout (str, optional) – The dropout used in the final layer of the network, ‘None’ or ‘dropout’. Default: ‘None’

get_seq()

class ecopann.sequence.Pooling

Bases: object

Pooling, to be used by classes LinearSeq & Conv2dSeq

class ecopann.sequence.SeqName(module_name)

Bases: object

The name of sequence, to be used by class LinearSeq.

seq_name()

ecopann.fcnet

class ecopann.fcnet.FcNet(node_in=2000, node_out=6, hidden_layer=3, nodes=None, activation_func='rrelu')

Bases: Module

Get a fully connected network.

Parameters

• node_in (int) – The number of the input nodes.

• node_out (int) – The number of the output nodes.

• hidden_layer (int) – The number of the hidden layers.

• nodes (None or list) – If list, it should be a collection of nodes of the network, e.g. [node_in, node_hidden1, node_hidden2, …, node_out]

• activation_func (str) – Activation function.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class ecopann.fcnet.MultiBranchFcNet(nodes_in=[100, 100, 20], node_out=6, branch_hiddenLayer=1, trunk_hiddenLayer=3, nodes_all=None, activation_func='rrelu')

Bases: Module

Get a multibranch network.

Parameters

• nodes_in (list) – The number of the input nodes for each branch. e.g. [node_in_branch1, node_in_branch2, …]

• node_out (int) – The number of the output nodes.

• branch_hiddenLayer (int) – The number of the hidden layers for the branch part.

• trunk_hiddenLayer (int) – The number of the hidden layers for the trunk part.

• nodes_all (list) – The number of nodes of the multibranch network. e.g. [nodes_branch1, nodes_branch2, …, nodes_trunk]

forward(x_all)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

ecopann.fcnet.split_nodes(nodes, weight=[])

ecopann.optimize

class ecopann.optimize.LrDecay(iter_mid, iteration=10000, lr=0.1, lr_min=1e-06)

Bases: object

Let the learning rate decay with iteration.

exp(gamma=0.999, auto_params=True)

Exponential decay.

Parameters

auto_params (bool) – If True, gamma is set automatically.

Returns

lr * gamma^iteration

Return type

float

poly(decay_step=500, power=0.999, cycle=True)

Polynomial decay.

Returns

(lr-lr_min) * (1 - iteration/decay_steps)^power +lr_min

Return type

float

step(stepsize=1000, gamma=0.3, auto_params=True)

Let the learning rate decays step by step, similar to ‘exp’.

ecopann.train

class ecopann.train.SimilarityIdx(param_nums)

Bases: object

simiIdx()

calculate the parameter index surrounding a specific parameter

simiIdx_2()

calculate the parameter index surrounding a specific parameter (using combination)

class ecopann.train.Train(net, loss_func='L1', iteration=10000, optimizer='Adam')

Bases: object

Train the network.

call_GPU(prints=True)

check_GPU()

train_0(xx, yy, iter_mid, repeat_n=3, lr_decay=True)

Training batch samples.

Parameters

• xx (torch tensor) – The input of the network.

• yy (torch tensor) – The target of the network.

• iter_mid (int) – The i-th iteration.

• repeat_n (int, optional) – The number of iterations using the same batch of data during network training, which is usually set to 1 or 3. Default: 3

• lr_decay (bool, optional) – If True, the learning rate will decrease with the iteration, otherwise, the learning rate will not change.

Returns

• float – The loss.

• Tensor – The predicted values.

train_1(inputs, target, repeat_n=1, set_seed=False, lr_decay=True, print_info=True, showIter_n=200)

Training the training set (or a subsample of the training set).

transfer_data(device=None)

transfer_net(use_DDP=False, device_ids=None, prints=True)

ecopann.train.additional_loss(predicted, yy, simiIdx, reduction='mean')

ecopann.train.additional_loss_2(predicted, yy, simiIdx, reduction='mean')

ecopann.train.loss_funcs(name='L1')

Some loss functions.

Parameters

name (str) – Abbreviation of loss function name. ‘L1’, ‘MSE’, or ‘SmoothL1’. Default: ‘L1’.

Returns

The corresponding loss function.

Return type

object

ecopann.models

class ecopann.models.LoadChain(path='ann', randn_num='0.123')

Bases: object

load_chain()

class ecopann.models.LoadHparams(path='ann', randn_num='0.123')

Bases: object

load_hparams()

class ecopann.models.LoadHparams_MB(path='ann', randn_num='0.123')

Bases: object

load_hparams()

class ecopann.models.LoadLoss(path='ann', randn_num='0.123')

Bases: object

load_loss()

class ecopann.models.LoadNet(path='ann', randn_num='0.123')

Bases: object

load_net()

class ecopann.models.MultiBranchMLP(train_set, param_names, vali_set=[None, None], obs_errors=None, cov_matrix=None, params_dict=None, branch_hiddenLayer=2, trunk_hiddenLayer=1, activation_func='rrelu', loss_func='L1', noise_type='multiNormal', factor_sigma=0.5, multi_noise=5)

Bases: OneBranchMLP

Multibranch network that is used to predict cosmological parameters with multiple sets of datasets.

Parameters

• train_set (list) – The training set that contains simulated observational spectra (data) which is a list spectra with shape [(N,spectra_length_1), (N,spectra_length_2), …] and simulated parameters of a specific cosmological (or theoretical) model. i.e. [spectra, parameters]

• param_names (list) – A list which contains the parameter names, e.g. [‘H0’,’Omega_m’,’ombh2’,’omch2’,’tau’,’As’,’ns’].

• vali_set (list, optional) – The validation set that contains simulated observational spectra (data) which is a list spectra with shape [(N,spectra_length_1), (N,spectra_length_2), …] and simulated parameters of a specific cosmological (or theoretical) model. The validation set can also be set to None. i.e. [spectra, parameters] or [None, None]

• obs_errors (list, optional) – Observational errors, it is a list of errors with shape [(spectra_length_1,), (spectra_length_2,), …]. Default: None

• cov_matrix (list or None, optional) – A list of covariance matrix with shape [(spectra_length_1, spectra_length_1), (spectra_length_2, spectra_length_2), …]. If there is no covariance for some observations, the covariance matrix should be set to None. e.g. [cov_matrix_1, None, cov_matrix_3]. Default: None

• params_dict (dict or None, optional) – Information of cosmological parameters that include the labels, the minimum values, and the maximum values. See params_dict_zoo(). Default: None

• branch_hiddenLayer (int, optional) – The number of the hidden layer for the branch part of the network. Default: 2

• trunk_hiddenLayer (int, optional) – The number of the hidden layer for the trunk part of the network. Default: 1

• activation_func (str, optional) – Activation function, which can be ‘relu’, ‘leakyrelu’, ‘prelu’, ‘rrelu’, ‘relu6’, ‘elu’, ‘celu’, ‘selu’, ‘silu’, ‘sigmoid’, ‘logsigmoid’, ‘tanh’, ‘tanhshrink’, ‘softsign’, or ‘softplus’ (see activation()). Default: ‘rrelu’

• loss_func (str, optional) – The loss function used in the network. Default: ‘L1’

• noise_type (str, optional) – The type of Gaussian noise added to the training set, ‘singleNormal’ or ‘multiNormal’. Default: ‘multiNormal’

• factor_sigma (float, optional) – For the case of ‘singleNormal’, it is the factor of the observational error (standard deviation), while for the case of ‘multiNormal’ it is the standard deviation of the coefficient of the observational error (standard deviation). Default: 0.5

• multi_noise (int, optional) – The number of realization of noise added to a spectrum. Default: 5

Variables

• spectra_base (array-like, optional) – The base value of spectra that is used for data normalization when training the network to ensure that the scaled spectra are ~ 1., it is suggested to set the mean of the simulated spectra. The default is the mean of the simulated spectra.

• params_base (array-like, optional) – The base value of parameters that is used for data normalization when training the network to ensure that the scaled parameters are ~ 1., it is suggested to set the mean of the posterior distribution (or the simulated parameters). The default is the mean of the simulated parameters.

• lr (float, optional) – The learning rate setting of the network. Default: 1e-2

• lr_branch (float, optional) – The learning rate setting of the branch part. Default: 1e-2

• lr_min (float, optional) – The minimum of the learning rate. Default: 1e-8

• batch_size (int, optional) – The batch size setting of the network. Default: 750

• auto_batchSize (bool, optional) – If True, the batch size will be set automatically in the training process, otherwise, use the setting of batch_size. Default: True

• epoch (int, optional) – The number of epoch of the training process. Default: 2000

• epoch_branch (int, optional) – The number of epoch for the branch part. This only works when training the branch part. Default: 2000

• base_epoch (int, optional) – The base number (or the minimum number) of epoch. Default: 1000

• auto_epoch (bool, optional) – If True, the epoch will be set automatically in the training process, otherwise, use the setting of epoch. Default: True

• scale_spectra (bool, optional) – If True, the input data (measurements) will be scaled based on the base values of the data. It is recommended to set to True. Default: True

• scale_params (bool, optional) – If True, the target data (cosmological parameters) will be scaled based on the base values of parameters. See ParamsScaling. It is recommended to set to True. Default: True

• norm_inputs (bool, optional) – If True, the input data of the network will be normalized. Default: True

• norm_target (bool, optional) – If True, the target data (cosmological parameters) will be normalized. Default: True

• independent_norm (bool, optional) – If True, the target data (cosmological parameters) will be normalized independently. This only works when norm_target is True. It is recommended to set to False. Default: False

• norm_type (str, optional) – The method of normalization, ‘z_score’, ‘minmax’, or ‘mean’ (see Normalize). Default: ‘z_score’

• spaceSigma_min (int, optional) – The minimum parameter space to be learned, e.g. for spaceSigma_min=5, the parameter space to be learned is \([-5\sigma, +5\sigma]\). Default: 5

• transfer_learning (bool, optional) – If True, the network will be initialized using the well-trained network of the previous step. Default: False

Note

It is suggested to set lr and lr_branch the same value.

predict(spectra, in_type='torch')

predict_chain(obs_data, cov_matrix=None, chain_leng=10000)

preprocessing_multiBranch_input(spectra, spectra_base, a=0, b=1)

preprocessing_multiBranch_target(params, a=0, b=1)

save_chain(path='ann', sample='TT')

save_hparams(path='ann', sample='TT')

save_loss(path='ann', sample='TT')

save_net(path='ann', sample='TT')

statistic()

train(repeat_n=3, showIter_n=100, train_branch=True, parallel=True, train_trunk=False, fix_lr=0.0001, reduce_fix_lr=False)

Train the network

Parameters

• repeat_n (int, optional) – The number of repeat feed to the network for each batch size data, which will increase the number of iterations in each epoch. Default: 3

• showIter_n (int, optional) – The number of epoch that show the training information. Default: 100

Returns

• object – The network object.

• array-like – The loss.

train_branch_trunk(repeat_n=3, showIter_n=100, train_branch_trunk=True)

transfer_branchNet(device=None)

transfer_data()

class ecopann.models.MultiDataSetMLP(spectra, parameters, param_names, obs_data, cov_matrix=None, params_dict=None, hidden_layer=3, activation_func='rrelu', loss_func='L1', noise_type='multiNormal', factor_sigma=0.5, multi_noise=5)

Bases: OneBranchMLP

predict(spectra, in_type='torch')

predict_chain(obs_data, cov_matrix=None, chain_leng=10000)

statistic()

train(repeat_n=3, showIter_n=100)

Train the network

Parameters

• repeat_n (int, optional) – The number of repeat feed to the network for each batch size data, which will increase the number of iterations in each epoch. Default: 3

• showIter_n (int, optional) – The number of epoch that show the training information. Default: 100

Returns

• object – The network object.

• array-like – The loss.

transfer_data()

class ecopann.models.OneBranchMLP(train_set, param_names, vali_set=[None, None], obs_errors=None, cov_matrix=None, params_dict=None, hidden_layer=3, activation_func='rrelu', loss_func='L1', noise_type='multiNormal', factor_sigma=0.5, multi_noise=5)

Bases: Train, ParamsScaling, CutParams

Multilayer perceptron (MLP) that is used to predict cosmological parameters with one set of datasets.

Parameters

• train_set (list) – The training set that contains simulated observational spectra (data) with shape (N, spectra_length) and simulated parameters of a specific cosmological (or theoretical) model. i.e. [spectra, parameters]

• param_names (list) – A list which contains the parameter names, e.g. [‘H0’,’Omega_m’,’ombh2’,’omch2’,’tau’,’As’,’ns’].

• vali_set (list, optional) – The validation set that contains simulated observational spectra (data) with shape (N, spectra_length) and simulated parameters of a specific cosmological (or theoretical) model. The validation set can also be set to None. i.e. [spectra, parameters] or [None, None]

• obs_errors (array-like, optional) – Observational errors with shape (spectra_length,). Default: None

• cov_matrix (array-like or None, optional) – Covariance matrix of the observational data. Default: None

• params_dict (dict or None, optional) – Information of cosmological parameters that include the labels, the minimum values, and the maximum values. See params_dict_zoo(). Default: None

• hidden_layer (int, optional) – The number of the hidden layer of the network. Default: 3

• activation_func (str, optional) – Activation function, which can be ‘relu’, ‘leakyrelu’, ‘prelu’, ‘rrelu’, ‘relu6’, ‘elu’, ‘celu’, ‘selu’, ‘silu’, ‘sigmoid’, ‘logsigmoid’, ‘tanh’, ‘tanhshrink’, ‘softsign’, or ‘softplus’ (see activation()). Default: ‘rrelu’

• loss_func (str, optional) – The loss function used in the network. Default: ‘L1’

• noise_type (str, optional) – The type of Gaussian noise added to the training set, ‘singleNormal’ or ‘multiNormal’. Default: ‘multiNormal’

• factor_sigma (float, optional) – For the case of ‘singleNormal’, it is the factor of the observational error (standard deviation), while for the case of ‘multiNormal’ it is the standard deviation of the coefficient of the observational error (standard deviation). Default: 0.5

• multi_noise (int, optional) – The number of realization of noise added to a spectrum. Default: 5

Variables

• spectra_base (array-like, optional) – The base value of spectra that is used for data normalization when training the network to ensure that the scaled spectra are ~ 1., it is suggested to set the mean of the simulated spectra. The default is the mean of the simulated spectra.

• params_base (array-like, optional) – The base value of parameters that is used for data normalization when training the network to ensure that the scaled parameters are ~ 1., it is suggested to set the mean of the posterior distribution (or the simulated parameters). The default is the mean of the simulated parameters.

• lr (float, optional) – The learning rate setting of the network. Default: 1e-2

• lr_min (float, optional) – The minimum of the learning rate. Default: 1e-8

• batch_size (int, optional) – The batch size setting of the network. Default: 750

• auto_batchSize (bool, optional) – If True, the batch size will be set automatically in the training process, otherwise, use the setting of batch_size. Default: True

• epoch (int, optional) – The number of epoch of the training process. Default: 2000

• base_epoch (int, optional) – The base number (or the minimum number) of epoch. Default: 1000

• auto_epoch (bool, optional) – If True, the epoch will be set automatically in the training process, otherwise, use the setting of epoch. Default: True

• scale_spectra (bool, optional) – If True, the input data (measurements) will be scaled based on the base values of the data. It is recommended to set to True. Default: True

• scale_params (bool, optional) – If True, the target data (cosmological parameters) will be scaled based on the base values of parameters. See ParamsScaling. It is recommended to set to True. Default: True

• norm_inputs (bool, optional) – If True, the input data of the network will be normalized. Default: True

• norm_target (bool, optional) – If True, the target data (cosmological parameters) will be normalized. Default: True

• independent_norm (bool, optional) – If True, the target data (cosmological parameters) will be normalized independently. This only works when norm_target is True. It is recommended to set to False. Default: False

• norm_type (str, optional) – The method of normalization, ‘z_score’, ‘minmax’, or ‘mean’ (see Normalize). Default: ‘z_score’

• spaceSigma_min (int, optional) – The minimum parameter space to be learned, e.g. for spaceSigma_min=5, the parameter space to be learned is \([-5\sigma, +5\sigma]\). Default: 5

• transfer_learning (bool, optional) – If True, the network will be initialized using the well-trained network of the previous step. Default: False

copyLayers_fromTrainedNet()

load_wellTrainedNet(path='ann', randn_num='0.123')

plot_loss(alpha=0.6, show_logLoss=False)

predict(spectra, use_GPU=False, in_type='torch')

predict_chain(obs_data, cov_matrix=None, chain_leng=10000, use_GPU=False)

predict_params(sim_spectra, use_GPU=False)

preprocessing_input(spectra, spectra_base, a=0, b=1)

preprocessing_target(params, a=0, b=1)

preprocessing_target_inv(params, a=0, b=1)

save_chain(path='ann', sample=None)

save_hparams(path='ann', sample=None)

save_loss(path='ann', sample=None)

save_net(path='ann', sample=None)

statistic()

train(repeat_n=3, showIter_n=100)

Train the network

Parameters

• repeat_n (int, optional) – The number of repeat feed to the network for each batch size data, which will increase the number of iterations in each epoch. Default: 3

• showIter_n (int, optional) – The number of epoch that show the training information. Default: 100

Returns

• object – The network object.

• array-like – The loss.

transfer_data()

class ecopann.models.RePredictMBMLP(path='ann', randn_num='0.123')

Bases: MultiBranchMLP, LoadNet, LoadLoss, LoadChain, LoadHparams_MB

Repredict cosmological parameters using the saved networks.

Parameters

• path (str) – The path of the results saved. Default: ‘ann’

• randn_num (str or int) – A random number that identifies the saved results.

class ecopann.models.RePredictOBMLP(path='ann', randn_num='0.123')

Bases: OneBranchMLP, LoadNet, LoadLoss, LoadChain, LoadHparams

Repredict cosmological parameters using the saved networks.

Parameters

• path (str) – The path of the results saved. Default: ‘ann’

• randn_num (str or int) – A random number that identifies the saved results.