Date: March 13, 2024

Topic: Core Intuitions

Recall

Notes

Core Intuition

• Outcomes affect decisions (if we know the outcome, we would make a better decision)
• Options affect decisions (changing an option can lead to a very different decision process)

Testing our code allows us to be more confident

Confidence (for coding)

How to be more confident in your solutions?

• Independently unit test each part
• Put it through a battery of sample problems (including corner cases)
• Validate that your tests and sample problems provide full code coverage
• Robust to input deviations

<aside> 📌 SUMMARY: The outcomes we get and the options we have will influence our decisions

</aside>

Date: March 13, 2024

Topic: Types of Actions

Recall

A robot’s movement is likely to be stochastic

Notes

Stochastic vs Deterministic

• Stochastic: The robot’s movements follow a probability distribution
  • 80% of the time going up, 20% going left or right
• Deterministic: A case of stochastic, where the movement is 100% in the desired direction
  • Always does the intended action

<aside> 📌 SUMMARY: A robot’s movement is likely to be stochastic, so we need to have an optimal policy for every location

</aside>

Date: March 13, 2024

Topic: Robustness

Recall

A robust solution should be able to handle many variations in the problem

Notes

Robustness

Solution is to output a converged value grid and an optimal policy

A robust solution can handle variations in:

• Action cost handle: different action costs
• Success probability: related to robot’s stochasticity, handle different distributions
• Wall cost: obstacles in the grid - add penalties to movement
• Goal location: should still be able to find goal after changing
• Grid layout: changing size of grid and varying it
• Allowable actions: 4-way vs 8-way

<aside> 📌 SUMMARY: A robust solution should be able to handle many variations in the problem

</aside>