System Design Cases

Machine Learning Basics

ML fundamentals: supervised/unsupervised/RL, train/val/test splits, overfitting and bias-variance, regularization, gradient descent, classical algorithms (linear/logistic, trees, RF, XGBoost) vs neural networks. Three scenarios: supervised pipeline, classification deploy, retrain on drift.

2 мин чтения·reference · free·1 интерактивная диаграмма

Machine learning: evidence from data under explicit assumptions

A model estimates behavior from a dataset and objective. Generalization depends on sampling, labels, leakage control, distribution, metric, threshold, and deployment feedback; training loss alone does not establish product value or safety.

Mental model

Cross-validation estimates generalization under assumptions about independent and identically distributed samples and must avoid preprocessing leakage. Choose group or time-aware splits that match deployment.
ML systems accumulate technical debt through data dependencies, feedback loops, undeclared consumers, and changing environments. Teach the surrounding system, not only the algorithm.
TFX describes production pipelines with data analysis, validation, training, model validation, and serving. Require artifact and data lineage across the lifecycle.
The ML Test Score proposes production-readiness tests beyond one offline metric. Use multi-dimensional gates and monitoring.

Guarantees and boundaries

The locked test set estimates performance only for its sampling process and metric definitions.
A model score is not a decision until a threshold and policy are applied.
Reproducibility requires data, code, environment, preprocessing, and randomness metadata.

Diagram scenarios

Leakage-aware training split. The split follows entities and time boundaries that will be independent at deployment.

Metrics follow the decision. Accuracy alone can hide class imbalance, subgroup harm, calibration, or asymmetric costs.

Leakage creates a false win. Future information, duplicate entities, or preprocessing fitted on all data can inflate evaluation.

Distribution and feedback shift. Production data and user behavior change, so an approved model can become unsuitable.

Architecture decision

Define the decision and harm model before training. Split by the unit that will be independent at deployment, preserve an untouched test set, compare against simple baselines, calibrate or choose thresholds on validation data, and monitor data plus outcome drift after release.

Failure modes

Data leakage can create high offline scores and poor deployment behavior.
Missing or delayed labels make online quality partially unobservable.
Feedback loops can change the population the model later learns from.

Operational checklist

Define prediction time, feature availability, entity unit, and label delay.
Compare with a simple non-ML or rules baseline.
Report uncertainty and slices, not only a single average.
Plan rollback and human handling before deployment.

[CONCEPT]mlops-pipeline

Primary sources

Machine Learning Basics·ML fundamentals: supervised/unsupervised/RL, train/val/test splits, overfitting and bias-variance, regularization, gradient descent, classical algorithms (linear/logistic, trees, RF, XGBoost) vs neural networks. Three scenarios: supervised pipeline, classification deploy, retrain on drift.open in editor ↗

Концепты в этой главе1

MLOps Pipeline

Спроси про эту главу

стр. 136 / 198

← Предыдущая глава

Fine-tuning vs RAG vs Prompting

Следующая глава →