MLOps
Streamline your ML workflow administration
![Towards Data Science](https://miro.medium.com/v2/resize:fill:48:48/1*CJe3891yB1A1mzMdqemkdg.jpeg)
Have you ever ever copy-pasted chunks of utility code between initiatives, leading to a number of variations of the identical code residing in several repositories? Or, maybe, you needed to make pull requests to tens of initiatives after the title of the GCP bucket by which you retailer your information was up to date?
Conditions described above come up approach too usually in ML groups, and their penalties differ from a single developer’s annoyance to the group’s incapability to ship their code as wanted. Fortunately, there’s a treatment.
Let’s dive into the world of monorepos, an structure broadly adopted in main tech firms like Google, and the way they will improve your ML workflows. A monorepo affords a plethora of benefits which, regardless of some drawbacks, make it a compelling alternative for managing complicated machine studying ecosystems.
We’ll briefly debate monorepos’ deserves and demerits, study why it’s a superb structure alternative for machine studying groups, and peek into how Massive Tech is utilizing it. Lastly, we’ll see methods to harness the facility of the Pants construct system to prepare your machine studying monorepo into a sturdy CI/CD construct system.
Strap in as we embark on this journey to streamline your ML undertaking administration.
This text was first printed on the neptune.ai weblog.
A monorepo (brief for monolithic repository) is a software program improvement technique the place code for a lot of initiatives is saved in the identical repository. The concept might be as broad as the entire firm code written in quite a lot of programming languages saved collectively (did anyone say Google?) or as slender as a few Python initiatives developed by a small group thrown right into a single repository.
On this weblog put up, we concentrate on repositories storing machine studying code.