> In a similar vein to general-to-specific transfer, we can compose new skills from existing policies—even without any fine-tuning—by intersecting different skills. The idea is simple: take two soft policies, each corresponding to a different set of behaviors, and combine them by adding together their Q-functions. In fact, it is possible to show that the combined policy is approximately optimal for the combined task, obtained by simply adding the reward functions of the constituent tasks, up to a bounded error.
> Because the maximum entropy formulation encourages agents to try all possible solutions, the agents learn to explore a large portion of the state space. Thus they learn to act in various situations, and are more robust against perturbations in the environment.
it is keeping non-zero weights on suboptimal actions during exploration. but that tells you almost nothing. there are an infinite number of ways to keep non-zero weights on suboptimal actions and some of those are much better than the rest. the maximum-entropy approach is a very specific way to use non-greedy weights.
> Because the maximum entropy formulation encourages agents to try all possible solutions, the agents learn to explore a large portion of the state space. Thus they learn to act in various situations, and are more robust against perturbations in the environment.
it is keeping non-zero weights on suboptimal actions during exploration. but that tells you almost nothing. there are an infinite number of ways to keep non-zero weights on suboptimal actions and some of those are much better than the rest. the maximum-entropy approach is a very specific way to use non-greedy weights.