MLflow 3.3.0 Bug: Experiments Comparison Broken

Aug 27, 2025 by RICHARD 48 views

MLflow 3.3.0 Bug: Comparing Experiments Not Working

Hey everyone! Have you been pulling your hair out trying to compare runs from different experiments in MLflow 3.3.0? Well, you're not alone! It seems like there's a nasty little bug that's been causing some headaches. In this article, we'll dive deep into the issue, explore the root cause, and show you how to reproduce it. We'll also talk about the steps to fix it, so you can get back to your machine learning experiments without any hiccups. Let's get started!

Understanding the MLflow Comparison Bug

So, what's the deal with this bug? Basically, when you're using MLflow 3.3.0, and you try to compare runs from multiple experiments, things don't go as planned. You'd expect to see a nice, organized page with all the runs side-by-side, ready for comparison. However, instead, you're unceremoniously dumped back at the MLflow homepage. Talk about a buzzkill! This is a real bummer, especially when you're trying to analyze and compare the performance of different models or configurations. It breaks the whole workflow and can be super frustrating. In this section, we are going to talk about what happens when you encounter this specific bug.

To reproduce the issue, you would need to get to the MLflow homepage. When you're there, the first thing you should do is create multiple experiments or make sure you already have some created. After doing so, you need to click on the experiments you want to compare. Usually, it's an easy click of the checkbox. After selecting the ones you want to compare, you need to click the Compare button. When you click on the button, the homepage will just show up, instead of the comparison page where all the experiments are. This is the main issue we are talking about. The expected outcome is to see a page that shows runs from all the experiments. However, the actual result is a redirection back to the home page. So, if you are using the tool and encounter this issue, you are not alone. This bug is real and has a fix that is also discussed in this article, so continue reading.

The root cause? A simple path route error. The application is being directed to the wrong place. The fix is a small change in the route-defs.ts file, as we'll see later. The bug affects the user experience. It prevents you from performing a crucial task: comparing experiments. This means that you can't easily see how different models or configurations stack up against each other. This is a major setback for anyone who relies on MLflow for experiment tracking and analysis. Because the compare feature is broken, it wastes time and requires manual workarounds, and slows down the model development cycle. It can lead to missed insights. The bottom line is, this bug can make it hard to get the most out of your experiments and make the best decisions about your machine learning models. Let's get into how to reproduce it and fix it, shall we?

Affected Version

The bug specifically impacts MLflow version 3.3.0. If you're using an older version, like 3.2.0, you're in the clear. The issue was introduced in the 3.3.0 release, so that's the version you need to be aware of. Upgrading to a newer version will likely fix the bug as well.

Steps to Reproduce the Bug in MLflow 3.3.0

Alright, let's get our hands dirty and walk through the steps to reproduce this bug. Follow these steps, and you'll see the issue firsthand. This will help you understand the problem and confirm that you're experiencing the same issue. You can also use these steps to check if a fix has been implemented. So, grab your favorite coding beverage and let's dive in!

Set Up the MLflow Server: First things first, you need to have an MLflow server up and running. We'll use Docker for this, making things super easy. Open up your terminal and run the following command:
```
# Run MLflow v3.3.0 at port 5000
docker run -itdq --name test -p 5000:5000 ghcr.io/mlflow/mlflow:v3.3.0 mlflow server --host 0.0.0.0
```
This command does a few things: It pulls the MLflow 3.3.0 Docker image, runs it in detached mode (-d), names the container test, and maps port 5000 on your local machine to port 5000 inside the container. This way, you can access the MLflow UI in your web browser. Now, you will have access to the MLflow UI at http://localhost:5000. Make sure the container is up and running before moving on!
Create Two Experiments: Now, let's create two experiments to compare. Open another terminal and execute the following commands to create two experiments named exp1 and exp2:
```
docker exec test mlflow experiments create -n exp1
docker exec test mlflow experiments create -n exp2
```
These commands use the docker exec command to run the mlflow experiments create command inside the Docker container. These commands will create two experiments. These are the experiments you'll be comparing later.
Access the MLflow UI: Open your web browser and go to http://localhost:5000/#/experiments. This will take you to the MLflow UI's experiments page. You should see the two experiments that you just created (exp1 and exp2), along with any other experiments you might have.
Reproduce the Bug: Now, it's time to trigger the bug. In the MLflow UI, select the checkboxes next to exp1 and exp2. Then, click the Compare button. Instead of being directed to a comparison page, you'll be redirected back to the experiments page. Boom! The bug is reproduced. You will realize that you are not able to compare the experiments. This is the core problem of this bug, and the main reason this article was created.
Clean Up: Once you're done testing, clean up the mess by stopping and removing the Docker container. Run the following command in your terminal:
```
docker stop test && docker rm test
```
This command stops the container named test and then removes it. This prevents any resources from being used unnecessarily.

Following these steps, you should be able to reproduce the bug and experience the frustration firsthand. It is also recommended to repeat the steps with ghcr.io/mlflow/mlflow:v3.2.0, and you will see that the bug will not be reproduced.

Understanding the Root Cause and the Fix

So, what's causing this annoying behavior? The issue lies in the routing configuration of the MLflow UI. The compare experiments functionality is pointing to the wrong component. This leads to the unexpected redirect back to the homepage. The core problem stems from an incorrect path route definition within the MLflow server's code. The compareExperimentsSearch route is incorrectly mapped, causing the application to load the homepage component instead of the desired comparison view. Now let's dive into the fix.

The fix is surprisingly simple. It involves modifying the route-defs.ts file in the MLflow server's codebase. Here's the code snippet that needs to be changed:

  {
    path: RoutePaths.compareExperimentsSearch,
-    element: createLazyRouteElement(() => import(/* webpackChunkName: "experimentPage" */ './components/HomePage')),
+    element: createLazyRouteElement(() => import(/* webpackChunkName: "experimentPage" */ './components/experiment-page/ExperimentPage')),
    pageId: PageId.compareExperimentsSearch,
  },

The original code was incorrectly mapping the compareExperimentsSearch route to the HomePage component. The corrected code maps it to the appropriate ExperimentPage component. This minor adjustment ensures that the compare experiments functionality works as expected. To implement this fix, you would need to modify the MLflow server's source code and rebuild the application. This typically involves cloning the MLflow repository, making the change, and then building the server. Then you just need to replace the file with the one that has the fix implemented.

By applying this fix, you ensure that when users click the 'Compare' button, they are directed to the intended comparison page, allowing them to effectively compare runs from selected experiments. The corrected routing directs users to the appropriate component responsible for displaying and managing experiment comparisons. The impact of this fix is that it immediately improves the user experience. The users can now easily compare runs, saving time and effort. This fix restores the intended functionality of MLflow's comparison feature and resolves the primary issue. The comparison page correctly displays the runs from the selected experiments, as expected.

Conclusion

And there you have it! We've explored the MLflow 3.3.0 bug that prevents you from comparing experiment runs, walked through the steps to reproduce it, and examined the simple fix. This bug is a minor inconvenience, but it is essential to know, especially if you are working with machine learning models. We've also discussed how to implement the fix, so you can get back to your machine learning experiments without any roadblocks. Remember, staying on top of these types of issues is crucial for maintaining a smooth and efficient workflow. Keep an eye out for updates and patches. If you encounter any other issues, don't hesitate to reach out to the MLflow community. Happy experimenting, and may your models always perform as expected!