MLOps

• ML Design Patterns
• ML System Design Pattern
• データセットシフト

Monitoring

shift detection

SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure | Papers With Code

Pipelines

paper: Towards Modular Machine Learning Pipelines

Towards Modular Machine Learning Pipelines - Microsoft Research

• Modi et al. (2023, July). Towards Modular Machine Learning Pipelines. In ICML Workshop on Localized Learning (LLW).

ML pipelinesがもつべき性質

1. Independently trainable: multiple components can be trained in parallel with very limited communication or coordination needed between them

2. Consistent: if a component is improved to its optimal version (i.e., replaced with the true data generating process for that component), the pipeline does not degrade

3. Aligned: if a component is incrementally improved, the pipeline is again guaranteed to not degrade.

   • Aligned pipelines may not be consistent — incremental shifts need not capture the large distribution shifts implied by consistency.

Data Quality-Driven

[2102.07750] A Data Quality-Driven View of MLOps

XユーザーのShinichi Takaŷanagi（減量中）さん: 「Microsoft Researchらによるデータ品質観点でのMLOps調査論文。MLOpsでの4つのタスク（MLモデル品質最適化、MLへの非現実的な期待防止、過学習の輿望、継続的な品質テスト）に対するhttps://t.co/eP2qHlDFJT を中心とした処方箋を紹介。 A Data Quality-Driven View of MLOps https://t.co/mw0fUkcogI」 / X