DAgger

Learning objectives Demonstrate the covariate shift problem: run the BC agent, record states it visits that were rare or absent in the expert data, and show that errors compound in those regions. Implement DAgger: collect new data by running the current BC policy (or a mix of expert and BC), query the expert for the correct action at those states, add to the dataset, and retrain BC. Explain why DAgger reduces covariate shift by adding on-policy (or mixed) states to the training set. Compare BC (trained only on expert data) with DAgger (iteratively aggregated) in terms of evaluation return and robustness. Relate covariate shift and DAgger to robot navigation and healthcare where the learner’s distribution can drift from the expert’s. Concept and real-world RL ...