XenorsMLflow Tutorial: Experiment Tracking and Model Registry
After ten experiments nobody remembers which settings produced the best model. MLflow records every run so results are comparable and reproducible.
What MLflow covers
MLflow has four parts: Tracking records runs, Projects package code, Models define a deployable format, and the Model Registry manages versions and stages or aliases. Most teams start with Tracking.
# pip install mlflow scikit-learn
# mlflow ui --port 5000 # run in a separate terminalTrack a training run
Log everything needed to reproduce the result: parameters, metrics, the model and a data version note.
import mlflow, mlflow.sklearn
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import f1_score
from sklearn.model_selection import train_test_split
mlflow.set_tracking_uri("http://127.0.0.1:5000")
mlflow.set_experiment("cancer-classifier")
X, y = load_breast_cancer(return_X_y=True)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
for n in (50, 200):
with mlflow.start_run(run_name=f"rf-{n}"):
model = RandomForestClassifier(n_estimators=n, random_state=42).fit(Xtr, ytr)
f1 = f1_score(yte, model.predict(Xte))
mlflow.log_param("n_estimators", n)
mlflow.log_metric("f1", f1)
mlflow.set_tag("data_version", "breast_cancer_v1")
mlflow.sklearn.log_model(model, "model")
print(n, round(f1, 4))Use autologging
For supported libraries, one line records parameters, metrics and the model automatically.
mlflow.sklearn.autolog()
with mlflow.start_run(run_name="auto"):
RandomForestClassifier(n_estimators=100).fit(Xtr, ytr)Register and load a model
Registering gives a model a name and version so deployment code can refer to it without knowing the run id.
import mlflow
from mlflow.tracking import MlflowClient
run_id = mlflow.search_runs(order_by=["metrics.f1 DESC"]).loc[0, "run_id"]
mv = mlflow.register_model(f"runs:/{run_id}/model", "cancer-classifier")
MlflowClient().set_registered_model_alias("cancer-classifier", "champion", mv.version)
model = mlflow.pyfunc.load_model("models:/cancer-classifier@champion")
print(model.predict(Xte[:3]))Team setup tips
For a shared server use a database backend such as PostgreSQL and object storage such as S3 for artifacts, rather than local files. Name experiments consistently, tag every run with a data version and Git commit, and never log secrets as parameters.
Hands-on practice
Run the examples in the downloadable Jupyter notebook. The extra checks below focus on mistakes that appear in real projects, not just syntax.
Log a parameter, metric and model artifact
import mlflow
with mlflow.start_run():
mlflow.log_param('max_depth',4)
mlflow.log_metric('f1',0.83)
mlflow.log_text('training complete','notes.txt')
print(mlflow.active_run().info.run_id)<run-id>
A useful run records enough context to reproduce the result, not only the final score.
Compare runs programmatically
runs = mlflow.search_runs(order_by=['metrics.f1 DESC'])
print(runs[['run_id','params.max_depth','metrics.f1']].head())run_id ... max_depth ... f1
Ranking runs is only useful when the validation split and metric definition are consistent.
Self-check
- Can you explain why the first practice result is trustworthy rather than just plausible?
- What would you monitor or test before using this MLflow Tracking workflow in production?
- Which assumption in the example would break first on a larger or messier dataset?
Frequently asked questions
Is MLflow free?
Yes. MLflow is open source. Managed versions are also available from some cloud platforms.
What should I log in every run?
Parameters, evaluation metrics, the model artifact, data version, code version and the environment or requirements file.
MLflow versus Weights & Biases?
Both track experiments. MLflow is open source and easy to self-host, while Weights & Biases is a hosted product with richer dashboards.
Primary references
Use these official sources when exact syntax, defaults or definitions matter beyond the examples here.
