Add Expert Functions

Introducing expert functions can provide domain-specific knowledge support for model training and inference, thereby improving the model’s expressive power and robustness, reducing task difficulty, and increasing model prediction accuracy. Expert functions can embed or invoke domain-specific knowledge in the model, enabling the model to quickly learn core issues and reduce model tuning time, thus lowering the task difficulty. Additionally, using predefined expert functions enhances the model’s accuracy as they have been carefully tested and used in production environments.

In mechanical control tasks, PID controllers are commonly used for control, which is a common control method. This control method can be defined as an expert function node to introduce domain-specific knowledge into model learning, accelerating model training and improving model accuracy. In autonomous driving tasks, traffic rules are a critical component. Traffic rules can be defined as expert function nodes.

The REVIVE SDK supports the introduction of expert functions. Domain-specific knowledge is useful when building complex virtual environments. The REVIVE SDK supports user-defined expert_function to introduce domain-specific knowledge into virtual environment model learning. If we know the calculation method of a node in the decision flowchart, we can define it as an expert function node using the keyword expert_function.

Here is an example definition of an expert function:

metadata:

   graph:
     action:
     - obs
     next_obs:
     - obs
     - action

   expert_functions:
     next_obs:
       'node_function' : 'dynamics.transition'

   ...
...

The configuration of expert functions is usually in the format of node_function:<filename>.<function_name>. The above .yaml file introduces an expert function named transition for the next_obs node, which should be defined in the dynamics.py file in the same directory.

The corresponding source code of the transition function is as follows:

import torch
from typing import Dict

def transition(data: Dict[str, torch.Tensor]) -> torch.Tensor:
   obs = data["obs"]
   return obs + 1

If a corresponding expert function is not bound to a node, REVIVE will automatically model the node through neural network.

Note

Any output node of the graph in the .yaml file supports binding expert functions, but at least one learnable node (i.e., initialized through neural network) must exist.

Here is an example of binding multiple expert function nodes, where the output of one expert function can be used as input for another expert function:

metadata:

   graph:
     action:
     - obs_1
     - obs_2
     next_obs_1:
     - obs_1
     - obs_2
     - action
     next_obs_2:
     - obs_1
     - obs_2
     - action
     - next_obs_1

   expert_functions:
     next_obs_1:
       'node_function' : 'dynamics.transition_1'
     next_obs_2:
       'node_function' : 'dynamics.transition_2'

   ...
...

When using expert functions to process data, it is important to organize multiple data into batches for processing, which can improve the efficiency of the code. Therefore, when writing reward functions, it is important to ensure that the function can handle multidimensional data corresponding to the input tensor shape. Additionally, when computing expert function outputs, the feature dimension of the last dimension is typically of interest.

For ease of processing, the computation dimension of the expert function is typically set to the last dimension. Therefore, when using the data, it is necessary to use slicing ([..., n:m ]) to obtain the feature of the last dimension of the data and perform calculations on it.

The output of the expert function should be a corresponding Pytorch Tensor, with the batch dimension remaining consistent with the input data, and the feature dimension of the last dimension should be consistent with the definition of the node in the *.yaml file.