revive.computation package¶
Submodules¶
revive.computation.dists module¶
- revive.computation.dists.all_equal(iterable)[source]¶
Define a function to check if all elements in an iterable are equal
- revive.computation.dists.exportable_broadcast(tensor1: Tensor, tensor2: Tensor) Tuple[Tensor, Tensor] [source]¶
Broadcast tensors to the same shape using onnx exportable operators
- class revive.computation.dists.ReviveDistributionMixin[source]¶
Bases:
object
Define revive distribution API
- property mode¶
return the most likely sample of the distributions
- property std¶
return the standard deviation of the distributions
- class revive.computation.dists.ReviveDistribution(batch_shape: Size = (), event_shape: Size = (), validate_args: bool | None = None)[source]¶
Bases:
TorchDistribution
,ReviveDistributionMixin
- class revive.computation.dists.ExportableCategorical(probs=None, logits=None, validate_args=None)[source]¶
Bases:
Categorical
- class revive.computation.dists.DiagnalNormal(loc, scale, validate_args=False)[source]¶
Bases:
ReviveDistribution
- sample(sample_shape=())[source]¶
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
- rsample(sample_shape=())[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
- log_prob(sample)[source]¶
Returns the log of the probability density/mass function evaluated at value.
- Args:
value (Tensor):
- entropy()[source]¶
Returns entropy of distribution, batched over batch_shape.
- Returns:
Tensor of shape batch_shape.
- property mode¶
Returns the mode of the distribution.
- property std¶
return the standard deviation of the distributions
- class revive.computation.dists.TransformedDistribution(base_distribution, transforms, validate_args=None)[source]¶
Bases:
TransformedDistribution
- property mode¶
Returns the mode of the distribution.
- property std¶
- class revive.computation.dists.DiscreteLogistic(loc, scale, num, *, validate_args=False)[source]¶
Bases:
ReviveDistribution
Model discrete variable with Logistic distribution, inspired from: https://github.com/openai/vdvae/blob/main/vae_helpers.py
As far as I know, the trick was proposed in: Salimans, Tim, Andrej Karpathy, Xi Chen, and Diederik P. Kingma “Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications.” arXiv preprint arXiv:1701.05517 (2017).
- Parameters:
loc – Location parameter, assert it have been normalized to [-1, 1]
scale – Scale parameter.
num – Number of possible value for each dimension.
- arg_constraints = {'loc': Real(), 'scale': GreaterThan(lower_bound=0.0)}¶
- support = Real()¶
- has_rsample = True¶
- log_prob(value)[source]¶
Returns the log of the probability density/mass function evaluated at value.
- Args:
value (Tensor):
- sample(sample_shape=())[source]¶
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
- rsample(sample_shape=())[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
- cdf(value)[source]¶
Returns the cumulative density/mass function evaluated at value.
- Args:
value (Tensor):
- icdf(value)[source]¶
Returns the inverse cumulative density/mass function evaluated at value.
- Args:
value (Tensor):
- property mode¶
Returns the mode of the distribution.
- property std¶
return the standard deviation of the distributions
- class revive.computation.dists.Onehot(logits=None, validate_args=False)[source]¶
Bases:
OneHotCategorical
,TorchDistributionMixin
,ReviveDistributionMixin
Differentiable Onehot Distribution
- has_rsample = True¶
- rsample(sample_shape=())[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
- property mode¶
Returns the mode of the distribution.
- property std¶
return the standard deviation of the distributions
- class revive.computation.dists.GaussianMixture(locs, coord_scale, component_logits)[source]¶
Bases:
MixtureOfDiagNormals
,ReviveDistributionMixin
- property mode¶
Returns the mode of the distribution.
- property std¶
return the standard deviation of the distributions
- class revive.computation.dists.MixDistribution(dists)[source]¶
Bases:
ReviveDistribution
Collection of multiple distributions
- arg_constraints = {}¶
- sample(num=())[source]¶
Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.
- rsample(num=())[source]¶
Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.
- entropy()[source]¶
Returns entropy of distribution, batched over batch_shape.
- Returns:
Tensor of shape batch_shape.
- log_prob(x)[source]¶
Returns the log of the probability density/mass function evaluated at value.
- Args:
value (Tensor):
- property mode¶
Returns the mode of the distribution.
- property std¶
return the standard deviation of the distributions
- property dists_type¶
revive.computation.funs_parser module¶
- revive.computation.funs_parser.convert_operators(line)[source]¶
Convert operator
Example: add -> opt.add
- revive.computation.funs_parser.convert_columns(line)[source]¶
Convert column name to column index
Example: “obs_1” -> 1
- revive.computation.funs_parser.convert_nodes(line)[source]¶
Convert node to tensor
Example: obs[1,2] -> obs[…,[1,2]]
revive.computation.graph module¶
- class revive.computation.graph.DesicionNode(name: str, input_names: List[str], input_descriptions: List[Dict[str, Dict[str, Any]]])[source]¶
Bases:
object
An abstract node for making decisions
- node_type: str = None¶
- __call__(data: Dict[str, Tensor], *args, **kwargs) Tensor | ReviveDistribution [source]¶
Run a forward computation of this node. NOTE: The input data was transferred by self.processor. You can use self.processor.deprocess_torch(data) to get the original data.
- register_processor(processor: DataProcessor)[source]¶
register the global data processor to this node
- to(device: str) DesicionNode [source]¶
change the device of this node
- requires_grad_(mode: bool = False) DesicionNode [source]¶
change the requirement of gradient for this node
- train() DesicionNode [source]¶
set the state of this node to training
- eval() DesicionNode [source]¶
set the state of this node to evaluation
- class revive.computation.graph.NetworkDecisionNode(name: str, input_names: List[str], input_descriptions: List[Dict[str, Dict[str, Any]]])[source]¶
Bases:
DesicionNode
- node_type: str = 'network'¶
- initialize_network(input_dim: int | dict, output_dim: int, hidden_features: int, hidden_layers: int, backbone_type: str, dist_config: list, is_transition: bool = False, hidden_activation: str = 'leakyrelu', norm: str | None = None, transition_mode: str | None = None, obs_dim: int | None = None, input_dim_dict: dict = {}, *args, **kwargs)[source]¶
initialize the network of this node
- call(data: Dict[str, Tensor], *args, **kwargs) ReviveDistribution [source]¶
Run a forward computation of this node. NOTE: The input data was transferred by self.processor. You can use self.processor.deprocess_torch(data) to get the original data.
- to(device: str) DesicionNode [source]¶
change the device of this node
- requires_grad_(mode: bool = False) DesicionNode [source]¶
change the requirement of gradient for this node
- train() DesicionNode [source]¶
set the state of this node to training
- eval() DesicionNode [source]¶
set the state of this node to evaluation
- class revive.computation.graph.FunctionDecisionNode(name: str, input_names: List[str], input_descriptions: List[Dict[str, Dict[str, Any]]])[source]¶
Bases:
DesicionNode
- node_type: str = 'function'¶
- register_node_function(node_function: Callable[[Dict[str, ndarray]], ndarray] | Callable[[Dict[str, Tensor]], Tensor], node_function_type: str)[source]¶
- call(data: Dict[str, Tensor], *args, **kwargs) Tensor | ReviveDistribution [source]¶
NOTE: if there is any provided function defined in numpy, this process cannot maintain gradients
- class revive.computation.graph.DesicionGraph(graph_dict: Dict[str, List[str]], descriptions: Dict[str, List[Dict[str, Dict[str, Any]]]], fit, metric_nodes)[source]¶
Bases:
object
A collection of DecisionNodes
- property learnable_node_names: List[str]¶
A list of names for learnable nodes the graph
- register_processor(processor: DataProcessor)[source]¶
Register data processor to the graph and nodes
- get_node(node_name: str, use_target: bool = False) DesicionNode [source]¶
get the node by name
- compute_node(node_name: str, inputs: Dict[str, Tensor], use_target: bool = False, *args, **kwargs)[source]¶
compute the node by name
- get_relation_node_names() List[str] [source]¶
get all the nodes that related to the learning (network) nodes. NOTE: this is the default list if you have matcher and value functions.
- get_node_value_net_node_names() List[str] [source]¶
get all the nodes that related to the learning (network) nodes. NOTE: this is the default list if you have matcher and value functions.
- is_equal_venv(source_graph: DesicionGraph, policy_node) bool [source]¶
check if new graph shares the same virtual environments
- is_equal_structure(source_graph: DesicionGraph) bool [source]¶
check if new graph shares the same structure
- copy_graph_node(source_graph: DesicionGraph) bool [source]¶
try copy all the node from source graph
- get_leaf(graph: Dict[str, List[str]] | None = None) List[str] [source]¶
return the leaf of the graph in alphabet order
- sort_graph(graph_dict: dict) OrderedDict [source]¶
Sort arbitrary computation graph to the topological order
- to(device: str) DesicionGraph [source]¶
- requires_grad_(mode: bool = False) DesicionGraph [source]¶
- eval() DesicionGraph [source]¶
- reset() DesicionGraph [source]¶
reset graph, useful for stateful graph
revive.computation.inference module¶
- revive.computation.inference.find_key_precedings(graph: DesicionGraph, node_name: str, preceding_type: str = 'network') List [source]¶
- revive.computation.inference.get_matcher_input(batch_data: Batch, target_node_name: str, candidate_num: int, node_names: List[str], nodes_fit_dict: dict | None = None) Tensor [source]¶
return all data of node_names from batch_data Args:
batch_data (Batch): Batch of data node_names (List): list of node names to get data nodes_fit_index (Dict): dict of fixed index for nodel_names
- Return:
data to get
- class revive.computation.inference.VirtualEnvDev(graph: DesicionGraph, train_algo: str | None = None, info: dict = {}, **kwargs)[source]¶
Bases:
Module
- to(device)[source]¶
Moves and/or casts the parameters and buffers.
This can be called as
- to(device=None, dtype=None, non_blocking=False)[source]
- to(dtype, non_blocking=False)[source]
- to(tensor, non_blocking=False)[source]
- to(memory_format=torch.channels_last)[source]
Its signature is similar to
torch.Tensor.to()
, but only accepts floating point or complexdtype
s. In addition, this method will only cast the floating point or complex parameters and buffers todtype
(if given). The integral parameters and buffers will be moveddevice
, if that is given, but with dtypes unchanged. Whennon_blocking
is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.See below for examples.
Note
This method modifies the module in-place.
- Args:
- device (
torch.device
): the desired device of the parameters and buffers in this module
- dtype (
torch.dtype
): the desired floating point or complex dtype of the parameters and buffers in this module
- tensor (torch.Tensor): Tensor whose dtype and device are the desired
dtype and device for all parameters and buffers in this module
- memory_format (
torch.memory_format
): the desired memory format for 4D parameters and buffers in this module (keyword only argument)
- device (
- Returns:
Module: self
Examples:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> linear = nn.Linear(2, 2) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]]) >>> linear.to(torch.double) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]], dtype=torch.float64) >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1) >>> gpu1 = torch.device("cuda:1") >>> linear.to(gpu1, dtype=torch.half, non_blocking=True) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1') >>> cpu = torch.device("cpu") >>> linear.to(cpu) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16) >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble) >>> linear.weight Parameter containing: tensor([[ 0.3741+0.j, 0.2382+0.j], [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128) >>> linear(torch.ones(3, 2, dtype=torch.cdouble)) tensor([[0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
- infer_k_steps(states: List[Dict[str, ndarray]], deterministic: bool = True, clip: bool = True) List[Dict[str, ndarray]] [source]¶
- infer_one_step(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True, **kwargs) Dict[str, ndarray] [source]¶
- node_infer(node_name: str, state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
- forward(data: Dict[str, Tensor], deterministic: bool = True, clip: bool = False) Dict[str, Tensor] [source]¶
run the target node
- pre_computation(data: Dict[str, Tensor], deterministic: bool = True, clip: bool = False, policy_index: int = 0, disturb_node_list: Dict = {}, **kwargs) Dict[str, Tensor] [source]¶
run all the node before target node. skip if the node value is already available.
- filter_penalty(penalty_type: str, data: Dict[str, Tensor], sample_num: int, clip: bool = False, policy_index: int = 0, **kwargs) Dict[str, Tensor] [source]¶
run all the node after target node
- class revive.computation.inference.VirtualEnv(env_list: List[VirtualEnvDev])[source]¶
Bases:
object
- to(device)[source]¶
Move model to the device specified by the parameter.
Examples:
>>> venv_model.to("cpu") >>> venv_model.to("cuda") >>> venv_model.to("cuda:1")
- check_version()[source]¶
Check if the revive version of the saved model and the current revive version match.
- reset() None [source]¶
When using RNN for model training, this method needs to be called before model reuse to reset the hidden layer information.
- property target_policy_name: str¶
Get the target policy name.
- replace_policy(policy: PolicyModel) None [source]¶
Replace the target policy with the given policy.
- infer_k_steps(states: Dict[str, ndarray] | List[Dict[str, ndarray]], k: int | None = None, deterministic: bool = True, clip: bool = True) List[Dict[str, ndarray]] [source]¶
Generate k steps interactive data.
- Args:
- states:
a dict of initial input nodes
- k:
how many steps to generate
- deterministic:
if True, the most likely actions are generated; if False, actions are generated by sample. Default: True
- Return:
k steps interactive data dict
Examples:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> ten_step_output = venv_model.infer_k_steps(state, k=10)
- infer_one_step(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True, **kwargs) Dict[str, ndarray] [source]¶
Generate one step interactive data given action.
- Args:
- state:
a dict of input nodes
- deterministic:
if True, the most likely actions are generated; if False, actions are generated by sample. Default: True
- Return:
one step outputs
Examples:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> one_step_output = venv_model.infer_one_step(state)
- node_infer(node_name: str, state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
Generate one step interactive data given node_name.
- Args:
- state:
a dict of input nodes
- deterministic:
if True, the most likely actions are generated; if False, actions are generated by sample. Default: True
- Return:
one step node output
Examples:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> action_output = venv_model.node_infer("action", state)
- node_dist(node_name: str, state: Dict[str, ndarray]) Dict[str, ndarray] [source]¶
Generate one step interactive dist given node_name.
- Args:
- state:
a dict of input nodes
- deterministic:
if True, the most likely actions are generated; if False, actions are generated by sample. Default: True
- Return:
one step node output
Examples:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> action_output = venv_model.node_infer("action", state)
- export2onnx(onnx_file: str, verbose: bool = True)[source]¶
Exporting the model to onnx mode.
Reference: https://pytorch.org/docs/stable/onnx.html
- Args:
- onnx_file:
the onnx model file save path.
- verbose:
if True, prints a description of the model being exported to stdout. In addition, the final ONNX graph will include the field
doc_string`
from the exported model which mentions the source code locations formodel
.
- class revive.computation.inference.PolicyModelDev(nodes: List[DesicionNode])[source]¶
Bases:
Module
- to(device)[source]¶
Moves and/or casts the parameters and buffers.
This can be called as
- to(device=None, dtype=None, non_blocking=False)[source]
- to(dtype, non_blocking=False)[source]
- to(tensor, non_blocking=False)[source]
- to(memory_format=torch.channels_last)[source]
Its signature is similar to
torch.Tensor.to()
, but only accepts floating point or complexdtype
s. In addition, this method will only cast the floating point or complex parameters and buffers todtype
(if given). The integral parameters and buffers will be moveddevice
, if that is given, but with dtypes unchanged. Whennon_blocking
is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.See below for examples.
Note
This method modifies the module in-place.
- Args:
- device (
torch.device
): the desired device of the parameters and buffers in this module
- dtype (
torch.dtype
): the desired floating point or complex dtype of the parameters and buffers in this module
- tensor (torch.Tensor): Tensor whose dtype and device are the desired
dtype and device for all parameters and buffers in this module
- memory_format (
torch.memory_format
): the desired memory format for 4D parameters and buffers in this module (keyword only argument)
- device (
- Returns:
Module: self
Examples:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> linear = nn.Linear(2, 2) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]]) >>> linear.to(torch.double) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]], dtype=torch.float64) >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1) >>> gpu1 = torch.device("cuda:1") >>> linear.to(gpu1, dtype=torch.half, non_blocking=True) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1') >>> cpu = torch.device("cpu") >>> linear.to(cpu) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16) >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble) >>> linear.weight Parameter containing: tensor([[ 0.3741+0.j, 0.2382+0.j], [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128) >>> linear(torch.ones(3, 2, dtype=torch.cdouble)) tensor([[0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
- class revive.computation.inference.PolicyModel(policy_model_dev: PolicyModelDev, post_process: Callable[[Dict[str, ndarray], ndarray], ndarray] | None = None)[source]¶
Bases:
object
- to(device: str)[source]¶
Move model to the device specified by the parameter.
Examples:
>>> policy_model.to("cpu") >>> policy_model.to("cuda") >>> policy_model.to("cuda:1")
- check_version()[source]¶
Check if the revive version of the saved model and the current revive version match.
- reset()[source]¶
When using RNN for model training, this method needs to be called before model reuse to reset the hidden layer information.
- property target_policy_name: None¶
Get the target policy name.
- infer(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True, additional_info: Dict[str, ndarray] | None = None) ndarray [source]¶
Generate action according policy.
- Args:
- state:
a dict contain ALL the input nodes of the policy node
- deterministic:
if True, the most likely actions are generated; if False, actions are generated by sample. Default: True
- clip:
if True, The output will be cropped to the range set in the yaml file; if False, actions are generated by sample. Default: True
- additional_info:
a dict of additional info for post process
- Return:
action
Examples:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> action = policy_model.infer(state)
- export2onnx(onnx_file: str, verbose: bool = True)[source]¶
Exporting the model to onnx mode.
Reference: https://pytorch.org/docs/stable/onnx.html
- Args:
- onnx_file:
the onnx model file save path.
- verbose:
if True, prints a description of the model being exported to stdout. In addition, the final ONNX graph will include the field
doc_string`
from the exported model which mentions the source code locations formodel
.
revive.computation.inference_cn module¶
- class revive.computation.inference_cn.VirtualEnvDev(graph: DesicionGraph)[source]¶
Bases:
Module
- to(device)[source]¶
Moves and/or casts the parameters and buffers.
This can be called as
- to(device=None, dtype=None, non_blocking=False)[source]
- to(dtype, non_blocking=False)[source]
- to(tensor, non_blocking=False)[source]
- to(memory_format=torch.channels_last)[source]
Its signature is similar to
torch.Tensor.to()
, but only accepts floating point or complexdtype
s. In addition, this method will only cast the floating point or complex parameters and buffers todtype
(if given). The integral parameters and buffers will be moveddevice
, if that is given, but with dtypes unchanged. Whennon_blocking
is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.See below for examples.
Note
This method modifies the module in-place.
- Args:
- device (
torch.device
): the desired device of the parameters and buffers in this module
- dtype (
torch.dtype
): the desired floating point or complex dtype of the parameters and buffers in this module
- tensor (torch.Tensor): Tensor whose dtype and device are the desired
dtype and device for all parameters and buffers in this module
- memory_format (
torch.memory_format
): the desired memory format for 4D parameters and buffers in this module (keyword only argument)
- device (
- Returns:
Module: self
Examples:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> linear = nn.Linear(2, 2) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]]) >>> linear.to(torch.double) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]], dtype=torch.float64) >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1) >>> gpu1 = torch.device("cuda:1") >>> linear.to(gpu1, dtype=torch.half, non_blocking=True) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1') >>> cpu = torch.device("cpu") >>> linear.to(cpu) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16) >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble) >>> linear.weight Parameter containing: tensor([[ 0.3741+0.j, 0.2382+0.j], [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128) >>> linear(torch.ones(3, 2, dtype=torch.cdouble)) tensor([[0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
- infer_k_steps(states: List[Dict[str, ndarray]], deterministic: bool = True, clip: bool = True) List[Dict[str, ndarray]] [source]¶
- infer_one_step(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
- node_infer(node_name: str, state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
- forward(data: Dict[str, Tensor], deterministic: bool = True, clip: bool = False) Dict[str, Tensor] [source]¶
run the target node
- pre_computation(data: Dict[str, Tensor], deterministic: bool = True, clip: bool = False, policy_index: int = 0) Dict[str, Tensor] [source]¶
run all the node before target node. skip if the node value is already available.
- class revive.computation.inference_cn.VirtualEnv(env_list: List[VirtualEnvDev])[source]¶
Bases:
object
- to(device)[source]¶
切换模型所在设备,可以指定cpu或cuda。
示例:
>>> venv_model.to("cpu") >>> venv_model.to("cuda") >>> venv_model.to("cuda:1")
- property target_policy_name: str¶
获得策略节点的名称。
- replace_policy(policy: PolicyModel) None [source]¶
使用给定的策略节点模型代替当前的策略节点模型。
- infer_one_step(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
生成1步交互数据,1步表示决策流图完整的运行一遍。
- 参数:
- states:
包含初始输入节点数据的字典,初始的节点数据应包括决策流图的所有叶子节点。
- deterministic:
如果参数是True, 进行确定性的数据生成; 如果参数是False, 从分布中进行采样生成数据。 默认值: True
- 返回值:
字典,含有1步交互数据,key是节点名,value节点数据数组。
示例:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> one_step_output = venv_model.infer_one_step(state)
- infer_k_steps(states: Dict[str, ndarray] | List[Dict[str, ndarray]], k: int | None = None, deterministic: bool = True, clip: bool = True) List[Dict[str, ndarray]] [source]¶
生成k步交互数据,每一步表示决策流图完整的运行一遍。
- 参数:
- states:
包含初始输入节点数据的字典,初始的节点数据应包括决策流图的所有叶子节点。
- k:
正整数,如果是1,则返回一步的交互数据;如果是10,则决策流图迭代的运行10次,返回10次的数据。
- deterministic:
如果参数是True, 进行确定性的数据生成; 如果参数是False, 从分布中进行采样生成数据。 默认值: True
- 返回值:
字典,含有k步交互数据,key是节点名,value是含有k步该节点数据的数组。
示例:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> ten_step_output = venv_model.infer_k_steps(state, k=10)
- node_infer(node_name: str, state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True) Dict[str, ndarray] [source]¶
使用指定节点模型进行推理.
- 参数:
- state:
包含节点所有输入数据的字典。
- deterministic:
如果参数是True, 进行确定性的数据生成; 如果参数是False, 从分布中进行采样生成数据。 默认值: True。
- 返回值:
节点输出。
示例:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> action_output = venv_model.node_infer("action", state)
- export2onnx(onnx_file: str, verbose: bool = True)[source]¶
导出环境模型为onnx格式。
参考: https://pytorch.org/docs/stable/onnx.html
- 参数:
- onnx_file:
存储onnx模型的文件地址。
- verbose:
默认为True。 如果为True,打印导出到的模型的描述,最终的ONNX图将包括导出模型中的字段doc_string,其中提到model的源代码位置。.
- class revive.computation.inference_cn.PolicyModelDev(nodes: List[DesicionNode])[source]¶
Bases:
Module
- to(device)[source]¶
Moves and/or casts the parameters and buffers.
This can be called as
- to(device=None, dtype=None, non_blocking=False)[source]
- to(dtype, non_blocking=False)[source]
- to(tensor, non_blocking=False)[source]
- to(memory_format=torch.channels_last)[source]
Its signature is similar to
torch.Tensor.to()
, but only accepts floating point or complexdtype
s. In addition, this method will only cast the floating point or complex parameters and buffers todtype
(if given). The integral parameters and buffers will be moveddevice
, if that is given, but with dtypes unchanged. Whennon_blocking
is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.See below for examples.
Note
This method modifies the module in-place.
- Args:
- device (
torch.device
): the desired device of the parameters and buffers in this module
- dtype (
torch.dtype
): the desired floating point or complex dtype of the parameters and buffers in this module
- tensor (torch.Tensor): Tensor whose dtype and device are the desired
dtype and device for all parameters and buffers in this module
- memory_format (
torch.memory_format
): the desired memory format for 4D parameters and buffers in this module (keyword only argument)
- device (
- Returns:
Module: self
Examples:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> linear = nn.Linear(2, 2) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]]) >>> linear.to(torch.double) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]], dtype=torch.float64) >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1) >>> gpu1 = torch.device("cuda:1") >>> linear.to(gpu1, dtype=torch.half, non_blocking=True) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1') >>> cpu = torch.device("cpu") >>> linear.to(cpu) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16) >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble) >>> linear.weight Parameter containing: tensor([[ 0.3741+0.j, 0.2382+0.j], [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128) >>> linear(torch.ones(3, 2, dtype=torch.cdouble)) tensor([[0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
- class revive.computation.inference_cn.PolicyModel(policy_model_dev: PolicyModelDev, post_process: Callable[[Dict[str, ndarray], ndarray], ndarray] | None = None)[source]¶
Bases:
object
- to(device: str)[source]¶
切换模型所在设备,可以指定cpu或cuda。
示例:
>>> policy_model.to("cpu") >>> policy_model.to("cuda") >>> policy_model.to("cuda:1")
- property target_policy_name: None¶
获得策略节点的名称。
- infer(state: Dict[str, ndarray], deterministic: bool = True, clip: bool = True, additional_info: Dict[str, ndarray] | None = None) ndarray [source]¶
使用策略模型进行推理,输出动作.
- 参数:
- state:
包含策略节点所有输入数据的字典
- deterministic:
如果参数是True, 进行确定性的数据生成; 如果参数是False, 从分布中进行采样生成数据。 默认值: True。
- clip:
如果为True,输出的动作数值将会被裁剪到YAML文件中配置的范围; 如果为False,不对输出的动作数值进行裁剪,输出值可能存在越界的情况。 默认值:True
- additional_info:
默认为None即可。
- 返回值:
动作。
示例:
>>> state = {"obs": obs_array, "static_obs": static_obs_array} >>> action = policy_model.infer(state)
- export2onnx(onnx_file: str, verbose: bool = True)[source]¶
导出策略模型为onnx格式。
参考: https://pytorch.org/docs/stable/onnx.html
- 参数:
- onnx_file:
存储onnx模型的文件地址。
- verbose:
默认为True。 如果为True,打印导出到的模型的描述,最终的ONNX图将包括导出模型中的字段doc_string,其中提到model的源代码位置。.
revive.computation.modules module¶
- class revive.computation.modules.Swish(*args, **kwargs)[source]¶
Bases:
Module
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.Value_Net(input_dim_dict, value_hidden_features, value_hidden_layers, value_normalization, value_activation, *args, **kwargs)[source]¶
Bases:
Module
Initializes a vectorized linear layer instance. Args:
in_features (int): The number of input features. out_features (int): The number of output features. ensemble_size (int): The number of ensembles to use.
- forward(state: Tensor | Dict[str, Tensor]) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.ConvBlock(in_features: int, out_features: int, conv_time_step: int, output_activation: str = 'identity')[source]¶
Bases:
Module
Initializes a vectorized linear layer instance. Args:
in_features (int): The number of input features. out_features (int): The number of output features. ensemble_size (int): The number of ensembles to use.
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.VectorizedLinear(in_features: int, out_features: int, ensemble_size: int)[source]¶
Bases:
Module
Initializes a vectorized linear layer instance. Args:
in_features (int): The number of input features. out_features (int): The number of output features. ensemble_size (int): The number of ensembles to use.
- class revive.computation.modules.EnsembleLinear(input_dim: int, output_dim: int, num_ensemble: int, weight_decay: float = 0.0)[source]¶
Bases:
Module
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- to(device)[source]¶
Moves and/or casts the parameters and buffers.
This can be called as
- to(device=None, dtype=None, non_blocking=False)[source]
- to(dtype, non_blocking=False)[source]
- to(tensor, non_blocking=False)[source]
- to(memory_format=torch.channels_last)[source]
Its signature is similar to
torch.Tensor.to()
, but only accepts floating point or complexdtype
s. In addition, this method will only cast the floating point or complex parameters and buffers todtype
(if given). The integral parameters and buffers will be moveddevice
, if that is given, but with dtypes unchanged. Whennon_blocking
is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.See below for examples.
Note
This method modifies the module in-place.
- Args:
- device (
torch.device
): the desired device of the parameters and buffers in this module
- dtype (
torch.dtype
): the desired floating point or complex dtype of the parameters and buffers in this module
- tensor (torch.Tensor): Tensor whose dtype and device are the desired
dtype and device for all parameters and buffers in this module
- memory_format (
torch.memory_format
): the desired memory format for 4D parameters and buffers in this module (keyword only argument)
- device (
- Returns:
Module: self
Examples:
>>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> linear = nn.Linear(2, 2) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]]) >>> linear.to(torch.double) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1913, -0.3420], [-0.5113, -0.2325]], dtype=torch.float64) >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1) >>> gpu1 = torch.device("cuda:1") >>> linear.to(gpu1, dtype=torch.half, non_blocking=True) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1') >>> cpu = torch.device("cpu") >>> linear.to(cpu) Linear(in_features=2, out_features=2, bias=True) >>> linear.weight Parameter containing: tensor([[ 0.1914, -0.3420], [-0.5112, -0.2324]], dtype=torch.float16) >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble) >>> linear.weight Parameter containing: tensor([[ 0.3741+0.j, 0.2382+0.j], [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128) >>> linear(torch.ones(3, 2, dtype=torch.cdouble)) tensor([[0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j], [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
- class revive.computation.modules.MLP(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str | None = None, hidden_activation: str = 'leakyrelu', output_activation: str = 'identity')[source]¶
Bases:
Module
Multi-layer Perceptron
Args:
in_features : int, features numbers of the input
out_features : int, features numbers of the output
hidden_features : int, features numbers of the hidden layers
hidden_layers : int, numbers of the hidden layers
norm : str, normalization method between hidden layers, default : None
hidden_activation : str, activation function used in hidden layers, default : ‘leakyrelu’
output_activation : str, activation function used in output layer, default : ‘identity’
- class revive.computation.modules.VectorizedMLP(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, norm: str | None = None, hidden_activation: str = 'leakyrelu', output_activation: str = 'identity')[source]¶
Bases:
Module
Vectorized MLP
- Args:
in_features : int, features numbers of the input
out_features : int, features numbers of the output
hidden_features : int, features numbers of the hidden layers
hidden_layers : int, numbers of the hidden layers
norm : str, normalization method between hidden layers, default : None
hidden_activation : str, activation function used in hidden layers, default : ‘leakyrelu’
output_activation : str, activation function used in output layer, default : ‘identity’
- class revive.computation.modules.ResBlock(input_feature: int, output_feature: int, norm: str = 'ln')[source]¶
Bases:
Module
Initializes a residual block instance. Args:
input_feature (int): The number of input features to the block. output_feature (int): The number of output features from the block. norm (str, optional): The type of normalization to apply to the block. Default is ‘ln’ for layer normalization.
- class revive.computation.modules.VectorizedResBlock(in_features: int, out_features: int, ensemble_size: int, norm: str = 'ln')[source]¶
Bases:
Module
Initializes a vectorized linear layer instance. Args:
in_features (int): The number of input features. out_features (int): The number of output features. ensemble_size (int): The number of ensembles to use.
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.ResNet(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str = 'bn', output_activation: str = 'identity')[source]¶
Bases:
Module
Initializes a residual neural network (ResNet) instance.
- Args:
in_features (int): The number of input features to the ResNet. out_features (int): The number of output features from the ResNet. hidden_features (int): The number of hidden features in each residual block. hidden_layers (int): The number of residual blocks in the ResNet. norm (str, optional): The type of normalization to apply to the ResNet. Default is ‘bn’ for batch normalization. output_activation (str, optional): The type of activation function to apply to the output of the ResNet. Default is ‘identity’.
- class revive.computation.modules.VectorizedResNet(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, norm: str = 'bn', output_activation: str = 'identity')[source]¶
Bases:
Module
Initializes a residual neural network (ResNet) instance.
- Args:
in_features (int): The number of input features to the ResNet. out_features (int): The number of output features from the ResNet. hidden_features (int): The number of hidden features in each residual block. hidden_layers (int): The number of residual blocks in the ResNet. norm (str, optional): The type of normalization to apply to the ResNet. Default is ‘bn’ for batch normalization. output_activation (str, optional): The type of activation function to apply to the output of the ResNet. Default is ‘identity’.
- class revive.computation.modules.TAPE(d_model, max_len=200, scale_factor=1.0)[source]¶
Bases:
Module
- forward(x)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TAttention(d_model, nhead, dropout)[source]¶
Bases:
Module
- forward(x)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TsTransformer(in_features, out_features, hidden_layers, hidden_features, dropout=0.0)[source]¶
Bases:
Module
- forward(x)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.Transformer1D(in_features: int, out_features: int, transformer_features: int = 16, transformer_heads: int = 8, transformer_layers: int = 4)[source]¶
Bases:
Module
This is an experimental backbone.
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.Tokenizer(d_numerical: int, categories: List[int] | None, d_token: int, bias: bool)[source]¶
Bases:
Module
- category_offsets: Tensor | None¶
- property n_tokens: int¶
- forward(x_num: Tensor, x_cat: Tensor | None)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.MultiheadAttention(d: int, n_heads: int, dropout: float, initialization: str)[source]¶
Bases:
Module
- forward(x_q, x_kv, key_compression: Linear | None, value_compression: Linear | None)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.FT_Transformer(in_features: int, out_features: int, hidden_features: int = 256, hidden_layers: int = 5, categories: List[int] | None = None, token_bias: bool = True, n_heads: int = 8, d_ffn_factor: float = 1.33, attention_dropout: float = 0.0, ffn_dropout: float = 0.0, residual_dropout: float = 0.0, activation: str = 'reglu', prenormalization: bool = True, initialization: str = 'kaiming', kv_compression: float | None = None, kv_compression_sharing: str | None = None)[source]¶
Bases:
Module
References: - https://github.com/Yura52/tabular-dl-revisiting-models/blob/main/bin/ft_transformer.py
- forward(x_num, x_cat=None)[source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.DistributionWrapper(distribution_type: str = 'normal', **params)[source]¶
Bases:
Module
wrap output of Module to distribution
- BASE_TYPES = ['normal', 'gmm', 'onehot', 'discrete_logistic']¶
- SUPPORTED_TYPES = ['normal', 'gmm', 'onehot', 'discrete_logistic', 'mix']¶
- forward(x: Tensor, adapt_std: Tensor | None = None, payload: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Warp the given tensor to distribution
:param adapt_std : it will overwrite the std part of the distribution (optional) :param payload : payload will be applied to the output distribution after built (optional)
- class revive.computation.modules.FeedForwardPolicy(in_features: int | dict, out_features: int, hidden_features: int, hidden_layers: int, dist_config: list, norm: str | None = None, hidden_activation: str = 'leakyrelu', backbone_type: str | str_ = 'mlp', use_multihead: bool = False, **kwargs)[source]¶
Bases:
Module
Policy for using mlp, resnet and transformer backbone.
- Args:
in_features : The number of input features, or a dictionary describing the input distribution out_features : The number of output features hidden_features : The number of hidden features in each layer hidden_layers : The number of hidden layers dist_config : A list of configurations for the distributions to use in the model norm : The type of normalization to apply to the input features hidden_activation : The activation function to use in the hidden layers backbone_type : The type of backbone to use in the model use_multihead : Whether to use a multihead model use_feature_embed : Whether to use feature embedding
- forward(state: Tensor | Dict[str, Tensor], adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.RecurrentPolicy(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, backbone_type: str | str_ = 'gru', **kwargs)[source]¶
Bases:
Module
Initializes a recurrent policy network instance.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features in the RNN. hidden_layers (int): The number of layers in the RNN. dist_config (list): The configuration for the distributions used in the model. backbone_type (Union[str, np.str_], optional): The type of RNN to use (‘gru’ or ‘lstm’). Default is ‘gru’.
- forward(x: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TsRecurrentPolicy(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, backbone_type: str | str_ = 'gru', ts_input_names: dict = {}, bidirectional: bool = True, dropout: float = 0, **kwargs)[source]¶
Bases:
Module
- input_is_dict = True¶
Initializes a recurrent policy network instance for ts_node.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features in the RNN. hidden_layers (int): The number of layers in the RNN. dist_config (list): The configuration for the distributions used in the model. backbone_type (Union[str, np.str_], optional): The type of RNN to use (‘gru’ or ‘lstm’). Default is ‘gru’.
- forward(x: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TsTransformerPolicy(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, ts_input_names: dict = {}, dropout: float = 0, **kwargs)[source]¶
Bases:
Module
- input_is_dict = True¶
Initializes a recurrent policy network instance for ts_node.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features in the RNN. hidden_layers (int): The number of layers in the RNN. dist_config (list): The configuration for the distributions used in the model. backbone_type (Union[str, np.str_], optional): The type of RNN to use (‘gru’ or ‘lstm’). Default is ‘gru’.
- forward(x: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.RecurrentRESPolicy(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, dist_config: list, backbone_type: str | str_ = 'gru', rnn_hidden_features: int = 64, window_size: int = 0, **kwargs)[source]¶
Bases:
Module
- forward(x: Tensor, adapt_std: Tensor | None = None, field: str = 'mail', **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.ContextualPolicy(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, dist_config: list, backbone_type: str | str_ = 'contextual_gru', **kwargs)[source]¶
Bases:
Module
- forward(x: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.EnsembleFeedForwardPolicy(in_features: int | dict, out_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, dist_config: list, norm: str | None = None, hidden_activation: str = 'leakyrelu', backbone_type: str | str_ = 'EnsembleRES', use_multihead: bool = False, **kwargs)[source]¶
Bases:
Module
Policy for using mlp, resnet and transformer backbone.
- Args:
in_features : The number of input features, or a dictionary describing the input distribution out_features : The number of output features hidden_features : The number of hidden features in each layer hidden_layers : The number of hidden layers dist_config : A list of configurations for the distributions to use in the model norm : The type of normalization to apply to the input features hidden_activation : The activation function to use in the hidden layers backbone_type : The type of backbone to use in the model use_multihead : Whether to use a multihead model use_feature_embed : Whether to use feature embedding
- forward(state: Tensor | Dict[str, Tensor], adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.FeedForwardTransition(in_features: int | dict, out_features: int, hidden_features: int, hidden_layers: int, dist_config: list, norm: str | None = None, hidden_activation: str = 'leakyrelu', backbone_type: str | str_ = 'mlp', mode: str = 'global', obs_dim: int | None = None, **kwargs)[source]¶
Bases:
FeedForwardPolicy
Initializes a feedforward transition instance.
- Args:
in_features (Union[int, dict]): The number of input features or a dictionary of input feature sizes. out_features (int): The number of output features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. dist_config (list): The configuration of the distribution to use. norm (Optional[str]): The normalization method to use. None if no normalization is used. hidden_activation (str): The activation function to use for the hidden layers. backbone_type (Union[str, np.str_]): The type of backbone to use. mode (str): The mode to use for the transition. Either ‘global’ or ‘local’. obs_dim (Optional[int]): The dimension of the observation for ‘local’ mode. use_feature_embed (bool): Whether to use feature embedding.
- forward(state: Tensor | Dict[str, Tensor], adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.RecurrentTransition(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, backbone_type: str | str_ = 'gru', mode: str = 'global', obs_dim: int | None = None, **kwargs)[source]¶
Bases:
RecurrentPolicy
Initializes a recurrent transition instance.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. dist_config (list): The distribution configuration. backbone_type (Union[str, np.str_]): The type of backbone to use. mode (str): The mode of the transition. Either ‘global’ or ‘local’. obs_dim (Optional[int]): The dimension of the observation.
- forward(state: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TsRecurrentTransition(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, backbone_type: str | str_ = 'gru', mode: str = 'global', obs_dim: int | None = None, **kwargs)[source]¶
Bases:
TsRecurrentPolicy
- input_is_dict = True¶
Initializes a recurrent transition instance.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. dist_config (list): The distribution configuration. backbone_type (Union[str, np.str_]): The type of backbone to use. mode (str): The mode of the transition. Either ‘global’ or ‘local’. obs_dim (Optional[int]): The dimension of the observation.
- forward(state: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.TsTransformerTransition(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, norm: str, dist_config: list, mode: str = 'global', obs_dim: int | None = None, ts_input_names: dict = {}, dropout: float = 0, **kwargs)[source]¶
Bases:
TsTransformerPolicy
- input_is_dict = True¶
Initializes a recurrent transition instance.
- Args:
in_features (int): The number of input features. out_features (int): The number of output features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. dist_config (list): The distribution configuration. backbone_type (Union[str, np.str_]): The type of backbone to use. mode (str): The mode of the transition. Either ‘global’ or ‘local’. obs_dim (Optional[int]): The dimension of the observation.
- forward(state: Tensor, adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.RecurrentRESTransition(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, dist_config: list, backbone_type: str | str_ = 'mlp', mode: str = 'global', obs_dim: int | None = None, rnn_hidden_features: int = 64, window_size: int = 0, **kwargs)[source]¶
Bases:
RecurrentRESPolicy
- forward(state: Tensor, adapt_std: Tensor | None = None, field: str = 'mail', **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.EnsembleFeedForwardTransition(in_features: int | dict, out_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, dist_config: list, norm: str | None = None, hidden_activation: str = 'leakyrelu', backbone_type: str | str_ = 'EnsembleRES', mode: str = 'global', obs_dim: int | None = None, **kwargs)[source]¶
Bases:
EnsembleFeedForwardPolicy
Initializes a feedforward transition instance.
- Args:
in_features (Union[int, dict]): The number of input features or a dictionary of input feature sizes. out_features (int): The number of output features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. dist_config (list): The configuration of the distribution to use. norm (Optional[str]): The normalization method to use. None if no normalization is used. hidden_activation (str): The activation function to use for the hidden layers. backbone_type (Union[str, np.str_]): The type of backbone to use. mode (str): The mode to use for the transition. Either ‘global’ or ‘local’. obs_dim (Optional[int]): The dimension of the observation for ‘local’ mode. use_feature_embed (bool): Whether to use feature embedding.
- forward(state: Tensor | Dict[str, Tensor], adapt_std: Tensor | None = None, **kwargs) ReviveDistribution [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.FeedForwardMatcher(in_features: int, hidden_features: int, hidden_layers: int, hidden_activation: str = 'leakyrelu', norm: str | None = None, backbone_type: str | str_ = 'mlp', **kwargs)[source]¶
Bases:
Module
Initializes a feedforward matcher instance.
- Args:
in_features (int): The number of input features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. hidden_activation (str): The activation function to use for the hidden layers. norm (str): The normalization method to use. None if no normalization is used. backbone_type (Union[str, np.str_]): The type of backbone to use.
- forward(x: Tensor) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.RecurrentMatcher(in_features: int, hidden_features: int, hidden_layers: int, backbone_type: str | str_ = 'gru', bidirect: bool = False)[source]¶
Bases:
Module
- forward(*inputs: List[Tensor]) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.HierarchicalMatcher(in_features: list, hidden_features: int, hidden_layers: int, hidden_activation: int, norm: str | None = None)[source]¶
Bases:
Module
- forward(*inputs: List[Tensor]) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.VectorizedMatcher(in_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, hidden_activation: str = 'leakyrelu', norm: str | None = None, backbone_type: str | str_ = 'mlp', **kwargs)[source]¶
Bases:
Module
Initializes a feedforward matcher instance.
- Args:
in_features (int): The number of input features. hidden_features (int): The number of hidden features. hidden_layers (int): The number of hidden layers. hidden_activation (str): The activation function to use for the hidden layers. norm (str): The normalization method to use. None if no normalization is used. backbone_type (Union[str, np.str_]): The type of backbone to use.
- forward(*inputs: List[Tensor]) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- class revive.computation.modules.EnsembleMatcher(matcher_record: Dict[str, deque], ensemble_size: int = 50, ensemble_choosing_interval: int = 1, config: dict | None = None)[source]¶
Bases:
object
- class revive.computation.modules.VectorizedCritic(in_features: int, out_features: int, hidden_features: int, hidden_layers: int, ensemble_size: int, norm: str | None = None, hidden_activation: str = 'leakyrelu', output_activation: str = 'identity')[source]¶
Bases:
VectorizedMLP
- class revive.computation.modules.Value_Net_VectorizedCritic(input_dim_dict, q_hidden_features, q_hidden_layers, num_q_net, *args, **kwargs)[source]¶
Bases:
Module
Initializes a vectorized linear layer instance. Args:
in_features (int): The number of input features. out_features (int): The number of output features. ensemble_size (int): The number of ensembles to use.
- forward(state: Tensor | Dict[str, Tensor]) Tensor [source]¶
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
revive.computation.operators module¶
revive.computation.utils module¶
- revive.computation.utils.maintain_gradient_hard_clamp(x: Tensor, _min=None, _max=None) Tensor [source]¶
- revive.computation.utils.get_input_dict_from_graph(graph, output_name: str, batch_data: Batch)[source]¶
- revive.computation.utils.get_sample_function(deterministic: bool) Callable[[ReviveDistribution], Tensor] [source]¶