1. 1. RMA Overall Structure Overview

The training is divided into two main phases: Phase 1 and Phase 2. Three key network modules are introduced as follows:

• Environmental Factor Encoder μ: (eₜ → zₜ)
  • Role: Used only during simulation training (Phase 1). Compresses "Privileged Information (eₜ)" available from the physics engine into a low-dimensional latent vector (zₜ).
• Base Policy π(xₜ, aₜ ₋ ₁, zₜ)
  • Role: Main Controller that controls the robot's joint positions. In Phase 1, it receives zₜ as input, and in real deployment (Phase 2 and Deployment), it receives ẑₜ as input.
• Adaptation Module φ: (history → ẑₜ)
  • Role: Instead of eₜ which is unknown in real deployment, estimates zₜ (as ẑₜ) by observing "past state and action history"

2. Environmental Factor Encoder μ

2.1. Paper Method

2.1.1. Input: Environment Vector eₜ (17D)

eₜ is a vector that quantifies environmental settings and is composed as follows:

• Mass + COM: 3D
• Motor Strength Scale: 12D (scale vector multiplied to each joint's position command before sending to actual PD target)
• Friction Coefficient: 1D (ground friction)
• Local Terrain Height: 1D (discretely summarized terrain height around the foot)
  • Specifically, the ground height at the current contact point of each of the four feet is read, expressed to the first decimal place, and only the maximum value among them is used. → In later papers (Egocentric), this seems to be used as Scandot

→ Total of 17D

2.1.2. Structure: 3-layer MLP

• μ: eₜ (17D) → zₜ (8D)
• Network Structure
  • Input: 17D environment vector eₜ
  • 1st hidden layer: 256
  • 2nd hidden layer: 128
  • Output: 8D extrinsics zₜ

In other words, μ is an encoder that learns "environment parameters → low-dimensional extrinsic representation suitable for the policy"

2.2. When Actually Examining the Code