Rémi Flamary

Domain Adaptation from shallow to deep learning tutorial

This is the page for the tutorial about Domain adaptation that I gave at the Peyresq signal and image processing summer school.

Course

The support of the course can be downloaded here: [PDF]

• Domain adaptation problem and generalization
  • Supervised learning, divergences and Optimal Transport
  • 50 shades of Data Shift
  • Generalization under data shift
  • The family of DA problems
• Classical Domain Adaptation methods
  • Reweighting methods
  • Subspace and alignment methods
  • Other approaches
  • Optimal Transport Domain Adaptation
• Deep Domain Adaptation
  • Domain invariant feature learning : one classifier to rule them all
  • Deep mapping approaches
  • Joint Distribution Optimal Transport (JDOT) and DeepJDOT
• Domain Adaptation variants
  • Multi-Source DA
  • Heterogeneous DA
• Domain Adaptation in Practice
  • How to validate with no labels ?
  • Reality check for DA

Bibliography

Domain Adaptation

• Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. (2009). Dataset shift in machine learning. The MIT Press.
• Moreno-Torres, J. G., Raeder, T., Alaiz-Rodrı́guez, R., Chawla, N. V., and Herrera, F. (2012). A unifying view on dataset shift in classification. Pattern recognition, 45(1):521–530.
• Kouw, W. M. and Loog, M. (2019). A review of domain adaptation without target labels. IEEE transactions on pattern analysis and machine intelligence, 43(3):766–785.

Theory of Domain Adaptation

• Ben-david, S., Blitzer, J., Crammer, K., and Pereira, O. (2006). Analysis of representations for domain adaptation. In Neural Information Processing Systems (NIPS). MIT Press.
• Redko, I., Morvant, E., Habrard, A., Sebban, M., and Bennani, Y. (2020a). A survey on domain adaptation theory. arXiv preprint arXiv:2004.11829.

Domain Adaptation and deep learning

• Csurka, G. (2017). A comprehensive survey on domain adaptation for visual applications. Domain adaptation in computer vision applications, pages 1–35.
• Wilson, G. and Cook, D. J. (2020). A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST), 11(5):1–46.

Practical Domain Adaptation

• Sugiyama, M., Krauledat, M., and Müller, K.-R. (2007). Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5).
• You, K., Wang, X., Long, M., and Jordan, M. (2019). Towards accurate model selection in deep unsupervised domain adaptation. In International Conference on Machine Learning, pages 7124–7133. PMLR.
• Musgrave, K., Belongie, S., and Lim, S.-N. (2021). Unsupervised domain adaptation: A reality check. arXiv preprint arXiv:2111.15672.