You think running a maze is hard? How about finding the exit inside you.
Overview
You’re given a pre-trained reinforcement learning (RL) agent (a DQN) that navigates a 12×12 maze. Under normal play, it moves from the start (0,0) to the goal (11,11) while avoiding a forbidden state at (6,6).
Your mission: exploit flaws in the agent and/or environment to force the agent into (6,6), which reveals the flag.
- Challenge Type: RL exploitation / reward hacking
- Difficulty: Hard
- Flag:
IEEE{rl_reward_hacking_forbidden_6_6}
Phase 1 — Understand the Maze Environment
What to do (explanation only)
- State space: 12×12 grid (144 positions).
- Action space: 4 moves (Up=0, Right=1, Down=2, Left=3).
- Rewards:
0.1 per step
+100 when reaching the goal (11,11)
- Special quirks: “bonus” rewards when certain hidden sequences occur.
- Forbidden state:
(6,6) contains the flag, but the agent avoids it under normal policy.
Key observations
- Reward shaping flaws → Circular movement patterns yield unintended cumulative rewards.
- Policy gaps → The agent was undertrained near
(6,6).
- Environment exploits → Specific action sequences let you phase through walls.
Phase 2 — Analyze Agent Behaviour