MLflow proved to have many limitations that neptune.ai can deal with, offering higher safety, extra strong collaboration instruments, and a user-friendly interface.
The migration shouldn’t be as complicated as you may assume. neptune.ai developed options to ease this course of.
Your MLflow run logs can simply be exported to the neptune.ai app utilizing a devoted plugin.
Use our MLflow vs neptune.ai API comparability desk emigrate your coaching scripts sooner.
As an MLflow consumer, it’s easy to adapt to neptune.ai’s UI.
MLflow is a framework broadly used for its experiment-tracking capabilities, however many organizations are looking for alternate options. It’s usually used as an early monitoring answer that groups can simply undertake as it’s well-known and open-source. Nonetheless, they usually find yourself needing extra superior options and higher help than MLflow can present.
MLflow lacks strong safety, collaboration, scaling, and interface modularity options. neptune.ai is a robust different to MLflow because it addresses these limitations. It’s constructed to handle bigger ML initiatives and affords a completely managed service to summary away the necessity for infrastructure upkeep.
This hands-on information will stroll you thru all the mandatory steps to carry out the migration. By the tip of this text, you’ll have transferred your knowledge, tailored your coaching scripts, and have all the knowledge you must accustom your staff to the brand new UI.
Seven causes for migrating from MLflow to neptune.ai
Earlier than diving into the migration course of, let’s see how MLflow and Neptune differ and why migrating from MLflow to Neptune is definitely worth the effort.
Safety and compliance: MLflow has restricted permissions administration and lacks built-in authentication options. Neptune affords superior entry controls and meets strict compliance requirements, together with SOC 2 certification, which is essential for organizations working with delicate knowledge.
Consumer entry administration: MLflow doesn’t help the administration of consumer entry and useful resource possession, whereas Neptune does.
Group collaboration: MLflow doesn’t present collaboration options comparable to reviewing initiatives, sharing knowledge, or creating detailed reviews. Neptune offers utilities that make this course of simpler with customizable workspaces and protracted shareable hyperlinks.
Consumer interface: Groups might discover MLflow’s interface much less adaptable for analyzing experiments. Neptune’s interface is richer, customizable, and extra intuitive, making it simpler to handle and assessment experiments.
Infrastructure upkeep: Self-hosting open-source MLflow entails configuring and managing the server supporting the appliance. Then again, Neptune is accessible as a completely managed instrument, saving your staff members valuable time.
Scalability and efficiency: MLflow shouldn’t be designed to deal with more and more giant knowledge volumes produced by your increasing coaching pipelines. Neptune can course of giant streams of logs working 1000s of experiments without delay.
Assist: Whereas MLflow’s neighborhood is supportive, there aren’t any ensures that your query or problem might be addressed. Neptune affords devoted consumer help, serving to to unravel points shortly.
Pc-vision fashions are an integral a part of Veo’s merchandise. Their sports activities cameras produce 4K video knowledge that’s analyzed to offer insights for coaches and gamers.
Initially, the staff began with MLflow because the experiment tracker however shortly discovered it unreliable, particularly beneath heavy computational hundreds. This led to connectivity points and crashes, disrupting their product improvement and typically inflicting them to lose per week’s price of labor.
Switching to Neptune not solely offered a dependable platform for his or her ML experiments but additionally allowed them to trace and analyze tens of metrics effortlessly.
Challenges when migrating between machine studying experiment trackers
Migrating from one experiment tracker to a different might sound overwhelming at first. Nonetheless, breaking it down into clear steps makes migrating from MLflow to Neptune lots much less daunting.
These are the important duties you’ll need to do:
Exporting coaching metadata: Step one is to export your present coaching logs from MLflow to Neptune. Neptune created the neptune-mlflow plugin to deal with this course of for you.
Copy dashboards: To keep up your experiment-tracking workflow and knowledge visibility, you’ll need to copy your MLflow dashboards within the new instrument. Neptune affords no-code dashboard options. You’ll be able to add pre-built widgets with drag-and-drop and resize to arrange your knowledge visualization as you would like.
Adapting coaching code: It’s important to modify your present MLflow coaching scripts to redirect run logs to the brand new instrument. You’ll be able to both redirect your MLflow logs to Neptune utilizing the neptune-mlflow plugin or carry out a full migration to Neptune’s native shopper.
Organising consumer accounts: To handle collaboration on the brand new instrument, you’ll need to create accounts and arrange consumer permissions. Neptune’s documentation affords steerage on find out how to add collaborators to your workspace and handle consumer roles.
Onboarding: You’ll have to prepare your staff to make use of the brand new instrument successfully.
Within the the rest of this text, we’ll work by way of these steps intimately.
Tips on how to migrate from MLflow to Neptune: step-by-step information
Listed here are the steps we are going to carry out collectively:
Put together for migration: We are going to guarantee we’ve got entry to the mandatory assets to carry out the migration.
Create a neptune.ai account and undertaking: We are going to join Neptune, entry our workspace, and create a undertaking to arrange our experiments.
Arrange an area Python setting: We’re going to create a Python setting with the mandatory packages to carry out the migration.
Configure the Neptune shopper: We are going to retrieve an API key and move it to the shopper together with the undertaking title.
Export MLflow logs to neptune.ai: We are going to migrate your present MLflow run logs and fashions to the Neptune experiment tracker.
Examine the imported knowledge: After the export has completed, we are going to confirm that each one knowledge has been transferred and may be accessed in Neptune.
Adapt your coaching scripts: We are going to adapt your coaching scripts to ship experiment metadata to Neptune by including only a few strains of Python code.
Non-obligatory: Migrate from the MLflow shopper to Neptune’s native shopper: We’re going to stroll by way of the refactoring of an MLflow-instrumented coaching script to make use of Neptune’s native shopper.
Put together for migration
Earlier than you begin, be sure that:
You’ve got a pc with Python and pip put in. (We developed this tutorial utilizing Python 3.10.12.) You will want to have the ability to create new Python environments and obtain software program packages from the web.
Your MLflow monitoring server is working and you understand its URI. (That is the deal with you move to your MLflow shopper by way of `mlflow.set_tracking_uri()` or the `MLFLOW_TRACKING_URI` setting variable.)
You’ll be able to entry the MLflow API from the pc you’re engaged on. (Should you can log knowledge to MLflow out of your pc, that is fulfilled.)
Create a Neptune account and undertaking
Should you wouldn’t have a Neptune account but, create one on the neptune.ai registration web page.
As soon as your account is created, entry your workspace by logging into the Neptune app and create a undertaking as follows:
Click on on `All initiatives` on the high left. Then click on on `Create undertaking` on the high proper.
The `Create new undertaking` kind will seem as proven beneath:
Select a reputation to your undertaking comparable to `mlflow-to-neptune`, and a undertaking key with higher case letters like `MIGRATION`. This key might be used to establish runs and different Neptune objects.
Lastly, click on the `Create` button on the backside proper. You now have a brand new, empty Neptune undertaking to switch your MLflow knowledge to.
Arrange an area Python setting
To carry out the migration, we might want to set up the Neptune and MLflow shoppers in a Python setting. We are going to construct the setting utilizing virtualenv to isolate it out of your different Python initiatives. (Should you’re already utilizing a special Python setting supervisor, be happy to make use of it as an alternative.)
First, set up `virtualenv` utilizing pip:
Should you retailer your MLflow knowledge regionally, navigate to the basis listing of your MLflow repository:
In all different instances, create a brand new listing (e.g., `mlflow_migration`).
Create a brand new digital setting:
Activate the setting you simply created:
You now have a purposeful Python setting.
Let’s proceed by putting in the mandatory packages to carry out the migration utilizing `pip`:
Configure the Neptune shopper
Subsequent, we have to configure the Neptune shopper. It wants the title and an entry token of the undertaking to which it ought to ship knowledge.
In your Neptune workspace, entry your Neptune API token as proven within the screenshot beneath:
Within the bottom-left nook of your display screen, click on in your username after which on `Get your API token`.
The next pop-up will present up:
Click on the `copy to clipboard` icon on the left to repeat your API token.
Then, set the `NEPTUNE_API_TOKEN` setting variable in your terminal:
Now, you must discover the exact identifier of your Neptune undertaking. On the principle web page of your workspace, click on in your undertaking. Then, click on on the `Edit undertaking particulars` possibility within the menu within the high proper:
The `Edit undertaking particulars` web page will seem.
Copy your undertaking identifier by clicking on the `copy to clipboard` icon:
Then, set the `NEPTUNE_PROJECT` variable to this identifier:
These two setting variables might be picked up by the Neptune shopper and permit it to entry your newly created undertaking.
Export MLflow logs to Neptune
Relying on whether or not you retailer your MLflow knowledge regionally or work with a distant monitoring server, the export works barely in another way.
Native MLflow
In your terminal, navigate to the listing containing your MLflow logs and run:
Should you don’t run the MLflow monitoring server at its default deal with, you’ll should move it by way of the `–mlflow-tracking-uri` command-line possibility:
Should you encounter a `NeptuneSynchronizationAlreadyStopped` exception whereas working `neptune mlflow`, don’t panic! You’ll merely must run the synchronization manually:
Distant MLflow
Should you retailer the MLflow knowledge remotely, e.g., utilizing a centralized MLflow monitoring server, you must move the monitoring URI to the export command:
The export plugin makes use of the MLflow monitoring shopper internally, so you possibly can move any type of URI supported by MLflow.
Examine the imported knowledge
In your Neptune undertaking, it is best to see all of the experiments beneath the `Run` tab. On the left, you’ll find an inventory of the experiments migrated from MLflow.
You may discover that the ID of the exported runs doesn’t match the run names in MLflow.
To match the run names, carry out these steps as proven within the screenshot above:
Click on on `Runs desk` on the high of the online web page.
Click on on `Add column` within the top-right nook.
Write `run_data/tags/mlflow.runName` within the search bar.
Click on on the merchandise `run_data/tags/mlflow.runName` beneath the search bar.
You now have a column in your customized view that shows the unique run names from MLflow:
Adapt your coaching scripts
We are able to use the identical neptune-mlflow plugin that we used for knowledge migration in our coaching script to instruct the MLflow shopper to log all knowledge to Neptune as an alternative of MLflow.
This lets you begin utilizing Neptune for all of your experiment monitoring shortly, with out having to switch the MLflow-specific capabilities comparable to `mlflow.log_metric()` or `mlflow.log_params()`.
Right here’s an instance of find out how to create a Neptune monitoring URI and move it to MLflow:
Non-obligatory: Migrate from the MLflow shopper to Neptune’s native shopper
Adapting your MLflow scripts utilizing the Neptune MLflow plugin is handy, because it redirects your logs to Neptune with minimal effort. Nonetheless, in the long term, you’ll most certainly wish to change to the Neptune shopper to get essentially the most out of Neptune and its versatile logging capabilities centered across the `Run` object.
MLflow makes use of a number of distinct capabilities to arrange experiment monitoring. In distinction, Neptune’s `init_run()` perform returns a `Run` object. This object is central to Neptune’s monitoring system and offers all of the strategies wanted to log info.
Data: Should you’re already aware of Neptune’s shopper, you possibly can skip the next instance and immediately bounce to the cheat sheet beneath.
Step-by-step instance: changing MLflow’s shopper with Neptune’s shopper
To information you thru the method of migrating to the Neptune shopper, we are going to refactor the MLflow quickstart script utilizing the neptune.ai API.
After working the refactored script, we are able to entry all of the uploaded metrics, parameters, and mannequin weights within the Neptune UI:
Cheat Sheet: Changing coaching scripts from the MLflow shopper to Neptune’s shopper
For extra info, learn the Neptune API and the MLflow API documentation.
Navigating Neptune as an MLflow consumer
Experiment trackers are fairly related, and switching from MLflow to Neptune is less complicated than you may assume.
Let’s stroll by way of Neptune’s interface, specializing in the similarities and variations to MLflow’s core options.
Experiments and initiatives
MLflow aggregates mannequin runs in `Experiments`. In Neptune, these are known as `Tasks`.
If you log into Neptune, your initiatives are within the heart of the workspace. That is much like how MLflow shows experiments within the left-hand sidebar:
Run listing view
Neptune and MLflow show the run listing view within the heart of the UI. That is the place you see all of your experiment runs at a look:
Neptune offers extra customization concerning which info to show:
Run particulars view
For each instruments, clicking on a row within the run listing takes you to the run particulars web page. That is the place you see all the knowledge associated to a particular experiment run:
Neptune affords a multi-pane view that shows the run listing, the logs, and the visualizations on one single web page. MLflow is much less versatile right here and solely shows the important thing info on the run particulars web page. To entry the visualizations, you must click on on a metric, which is able to open a brand new web page.
Mannequin listing view
In MLflow, you entry the mannequin view from the top-left nook. In Neptune, you attain this view from the left-hand sidebar:
MLflow combines all of the fashions from varied experiments. This setup may be complicated when many groups use the identical experiment tracker as a result of everybody’s information get combined collectively. Neptune organizes fashions into totally different initiatives, making it simpler to view solely your personal fashions.
Mannequin particulars view
For each trackers, clicking on a row within the mannequin listing view brings you to the mannequin particulars view:
That is the place you possibly can view the knowledge associated to a particular mannequin model.
Neptune helps logging detailed info for every mannequin model. You’ll be able to log coaching metrics and even the inference velocity on particular {hardware}. As an alternative, MLflow solely exhibits primary info, such because the enter and output schema, tags, and outline.
Conclusion
Migrating from MLflow to neptune.ai is less complicated than it appears. Neptune affords all of the options, documentation, and instruments to simplify transferring your experiment logs, adapting your coaching scripts, managing consumer accounts, and onboarding your staff.
Beginning the migration is easy. You simply want a pc with Python and guarantee you possibly can connect with your MLflow monitoring server by way of the MLflow API.
Right here’s a abstract of the principle steps to carry out the migration:
1
Create a Neptune account and a undertaking.
2
Create a Python setting and set up the required dependencies.
3
Export your MLflow logs to Neptune.
4
Adapt your coaching scripts or migrate to the Neptune native shopper.
Utilizing Neptune as an MLflow consumer is easy, as each instruments share many similarities. Adapting your experiment workflows requires little effort and yields nice advantages attributable to Neptune’s intensive enhancements.
We hope this information has helped you execute or plan your migration. If in case you have any additional questions or suggestions, please don’t hesitate to achieve out. We’re right here to help you in each step of the method.