Kit is an open source #MLOps project that packages your model, datasets, code, and configuration so data scientists and developers can use their preferred tools

Remote
We built #KitOps to simplify the way AI/ML artifacts are shared and managed throughout the lifecycle of a projects. A big pain point for developers and DevOps teams. We did this by building #ModelKits. As an OCI-compliant packaging format, a ModelKit encapsulates datasets, code, configurations, and models into a single, standardized unit. This approach not only streamlines the development process but also ensures broad compatibility and integration with a vast array of tools and platforms. Key features of ModelKits: Seamless Sharing and Collaboration: ModelKit's standardized format fosters a collaborative environment, enabling teams to share and manage AI/ML artifacts effortlessly across different stages of development. Wide Compatibility: Being OCI-compliant, ModelKits can be stored, versioned, and tagged using existing infrastructure like DockerHub or GitHub Packages, leveraging familiar workflows for AI/ML artifacts and streamlining infrastructure costs. Efficient Artifact Management: Unlike traditional container images, ModelKits allow for direct addressing of included artifacts. This means tools can selectively unpack only the required datasets or code at any given stage, optimizing resource usage and speeding up development. Enhanced Efficiency for Shared Artifacts: ModelKits are designed to efficiently handle shared artifacts across multiple versions. When the same dataset, for instance, are used by several ModelKits, this approach significantly reduces duplication and storage overhead. Built-in Versioning and Tagging: Leveraging existing container registry infrastructure, ModelKits support sophisticated versioning and tagging strategies out of the box, something that requires additional tooling or manual management with traditional storage. Optimized for AI/ML Workflows: ModelKits are tailor-made for AI/ML projects, addressing specific needs such as versioning and environment configuration. ModelKit is not just a packaging format; it's a building block for innovation, simplifying the complexities of AI/ML development and deployment. By adopting ModelKit, teams can focus more on creating value and less on managing the intricacies of artifact storage and sharing. You can learn more about KitOps and ModelKits here: KitOps.ml
2
14
101
8,087
Have you tried out the new KitOps Dev Mode? Dev Mode provides a UI for interacting with #LLMs on your laptop (no connection needed) and is one of the easiest ways to get started with prompt engineering. github.com/jozu-ai/kitops #LLM #AIML #MachineLearning #AIMLOps #LLMOps
3
15
313,134
We're excited to announce that the Kit Cli has been released into beta. Check it out. Learn more at kitops.ml #MLOps #CLI #DevTools #OpenSource
2
5
12
2,372
Fine-tune your first LLM today. Getting started with LLMs can be intimidating. This tutorial will walk you through fine-tuning a large language model using LoRA, facilitated by tools like llama.cpp and KitOps. dev.to/kitops/fine-tune-your…
2
10
243,799
Not sure where to start with your #MLOps pipeline? Here's a step-by-step guide to the full process. jozu.com/blog/a-step-by-step…
6
12
369
The friction in the #AIML development process is overwhelming. Jupyter notebooks are great ... and suck ... Containers, not optimal ... The divide between data scientist and DevOps is growing ... What do we do? Let's talk about it! dev.to/kitops/why-enterprise…
1
5
25,980
When working on AI Projects, the collaboration process between data scientist, application engineers, and DevOps/SRE teams can become bottlenecked or even strained. A lot of this has to do with the tools they are using and how compatible they are. In this post, we explore how KitOps eases handoffs and collaboration across teams. #MLOps #DevOps #OpenSource dev.to/kitops/tools-to-ease-…

ALT tug oar GIF

8
11
370
With KitOps, you can tune an #LLM in just 5 steps. Try it out! #LLaMA3 #Lora #MachineLearning dev.to/kitops/fine-tune-your…
2
8
97,595
Awesome post by @nevodavid on @ThePracticalDev this morning to get you started with hosting and tuning your own models. It covers how the team at @winglangio is building an #LLM to answer questions in their docs. #TLDR - they tuned an LLM with Autotrain by @huggingface (using almost zero python), then packaged it up with @Kit_Ops dev.to/github20k/i-fine-tune…
2
6
335
🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟 This weekend KitOps broke 200 stars on GitHub! 🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟🌟 Thanks for supporting our mission of easier AI project development. 🚀🚀🚀🚀🚀🚀 #KitOps #GitHubStars #SoftwareDevelopment #AIML
1
7
257
10 #OpenSource tools you can use to build an MLOps pipeline: KitOps (kitops.ml) - KitOps, treats all components in an ML project as a single software unit, making it easier to package, version, and track those components. This is called a ModelKit, and is based on the OCI-standard (similar to #Docker and #Kubernetes), which makes ModelKits compatible with almost every software development tool. With KitOps, it is possible to unpack individual components (data, model, or code) and work on them.
1
1
6
121
We reached 150 #GitHub stars this weekend! Thanks for the support everyone - We have an exciting release coming this week, keep an eye out 😉 Not familiar with the project? Check it out: github.com/jozu-ai/kitops We're making it way easier to work with your AI projects. #MLOps #LLMOps #DevOps
1
1
5
107
Moving models from dev to prod is a pivotal challenge. And, the hurdles, particularly when moving from test environments to real-world settings often go unaddressed. So what do you do? We explore these challenges on today's episode of #PartiallyRedacted open.spotify.com/episode/1Bo…
1
5
187
⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ THANK ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ YOU ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ FOR ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ ALL ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ THE ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ SUPPORT ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ 100 ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ GITHUB ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ STARS ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ TODAY!
1
5
449
At some point in the near future, you'll be handed a Jupyter Notebook and told that it's ready to go into production. Unfortunately, that's easier said than done. 1. Notebooks have a non-linear flow of logic and hidden state (variables that change within the notebook). This makes it difficult to understand how the code works and troubleshoot issues in production. 2. Notebooks are often single-threaded and not designed to handle high volumes of traffic. They don't perform well in applications requiring real-time responsiveness or work efficiently across multiple nodes or clusters. 3. Notebooks can execute arbitrary code and often contain sensitive information, raw code, and data, which can be a security vulnerability if not properly managed. 4. Logging, monitoring, and integration with other systems are common in production environments, and Notebooks are not built for this type of integration. We built #KitOps to solve these issues ... here's how: jozu.com/blog/how-to-turn-a-…
2
5
114
last week KitOps maintainer @BradMicklea joined @georgefirican on the #LightsOnData podcast to discuss ways to improve collaboration between enterprise development teams and AI/ML teams. You can watch the full conversation on YouTube: piped.video/watch?v=jRi3wmaH…
2
1
170
Security is one of the main reasons developers are adopting KitOps. Specifically KitOps ensures your projects are: Tamper-Proof: With KitOps, your artifacts are locked down, hashed, and immutably stored. No more waking up in cold sweats wondering if something was altered on the sly. Auditable: Every artifact comes with a complete history, ensuring that when the auditors come knocking, you’re not scrambling for answers. Tagged and Versioned: With built-in support for champion vs. challenger models, semantic versioning, and more, KitOps makes it easy to manage complex #ML workflows. Elegant Bundled: KitOps doesn’t just store your artifacts—it bundles them with all the metadata you need, ensuring that every deployment is consistent and reliable. Consistency Across the Supply Chain: By storing everything in #Artifactory, KitOps ensures that your AI/ML workflows are as seamless as the rest of your #DevOps processes.
3
4
146
A lot of dev teams are using #Docker to package their AI/ML projects. It works, but because Docker wasn't built for AI project development there are some limitations like: 1. Lack of version control for model and data 2. Bloated container images 3. Difficulty managing dependencies 4. Complexity of ML projects We built #ModelKits to solve these issues. But(!) that doesn't mean that you should stop using Docker. Here's the scoop: jozu.com/blog/when-to-docker…
1
4
309
What if you could deploy your models faster and more efficiently with a standards-based approach, ensuring compatibility and flexibility across tools and platforms, while simplifying project management with built-in versioning and tagging for your AI/ML artifacts? #KitOps
2
4
51
🧵 How do you cut AI/ML development time in half? How do you do it with #OpenSource? We've compiled a list of open source tools that will help you get your AI projects to production in half the time.
1
4
8
207
#sLLM's are quickly becoming a focal point for enterprise AI Projects. They can be tuned to specialized tasks, and are easier to work on than their larger counterparts. In this post, we walk you thoug how to tune and deploy your first small language model (sLLM) dev.to/kitops/how-to-tune-an…
4
4
165
🥳 75 @github stars, a new GitHub Action, and a feature in @thenewstack. It's been a great week 🚀 Thanks for the support everyone! #KitOps
2
4
304
What makes a good workflow between data scientists and DevOps or application engineering teams? In this post we explore #opensource tools that can be used to ease collaboration across teams and help get your ML projects to production faster. jozu.com/blog/tools-to-ease-…
4
51
KitOps Dev Mode is here 🚀🚀🚀 v0.2.0 has been released: This update brings two major features for working with #LLMs, as well numerous smaller enhancements. #LLM #MLOps #OpenSource #DevOps #LLMOps dev.to/kitops/kitops-release…
1
4
4
313
🚀 Last week had our first community PR on the KitOps.ml repo 🥰 It's exciting to see the support and contribution from the developer community. Thanks everyone! Let's build a better ML deployment process, together! #MLOps #DevOps #OpenSource
3
56
#KitOps V0.3.1 is now available Notable feature improvements include: -Resumable downloads when pulling ModelKits -New efficient storage layout for storing blobs -Notifications when newer versions of the CLI is available Also included in the release are multiple bug fixes and doc enhancements. See full release here: github.com/jozu-ai/kitops/re…
1
2
3
108
There's more to #KitOps than meets the eye 👀 Here are 13 Kit features, courtesy of @BradMicklea Did you know all of them ⁉️ 🧵
1
1
3
166
Did you catch our appearance on Adventures in Machine Learning? Streamline your MLOps with KitOps–Learn how KitOps helps to ease model handoffs, improve collaboration, and support model versioning. piped.video/watch?v=vttrjWPH…
1
3
91
Kit v0.1.1 has been released, it includes: - Progress bars for pack/unpack, and pull/push - Improvements to our documentation - Error message improvements To check it out or install the latest version of Kit visit: github.com/jozu-ai/kitops
1
3
437
Another day, another milestone! 🙏 Thanks for the love ♥️🥰♥️ 👀 Check out what we're working on here: kitops.ml ⭐️ Support Kit with a star: github.com/jozu-ai/kitops #KitOps #DevOps #CICD #AIML #OpenSource #GitHub
3
32
Another Friday, another #KitOps release 🚀 This release includes .kitignore file implementation, bug fixes 🪲 🔨, and documentation improvements. Checkout the full update here: github.com/jozu-ai/kitops/re…
2
2
378
Who is #KitOps for? 👩‍💻 For #AppDevs​ KitOps clears the path to use AI/ML with your existing tools and applications. No need to be an AI/ML expert, KitOps lets you concentrate on integrating AI/ML models into your applications, while KitOps handles the packaging and sharing. 👷 For #DevOps teams​ ModelKits fit into your existing processes and the Kit CLI lets you pack or unpack ModelKit artifacts in the pipelines and automations you have proven over the last decade. 👩‍🔬 For #datascientists​ KitOps enables you to innovate in AI/ML without the usual infrastructure distractions. It simplifies dataset and model management and sharing, fostering closer collaboration with developers. With KitOps, you can spend more time experimenting and less time grappling with traditional software development tools.
4
3
265
We built KitOps ModelKits to be OCI-compliant, making them compatible with your existing Docker registries. Visit kitops.ml #DevOps #MachineLearning #MLOps
1
3
122
KitOps V0.3.3 is now available 🚀 Most notably, this release features: One major bug fix–Upacking no longer hangs indefinitely when files already exist And, One feature release–advanced filtering (--filter=model,datasets) You can view the full release here: github.com/jozu-ai/kitops/re…
2
4
526
We built Kit for many reasons, one of them, saving the #DevOps team's hair. #MLOps #KitOps #AIML
3
3
85
We've joined the 30-star club (that's a thing right? 🤷‍♀️) ⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️⭐️ github.com/jozu-ai/kitops
3
428
Curious about building and distributing a sleek RAG pipeline? Checkout @GorkemErcan's tutorial on The Code Project. This tutorial walks you through the process of creating a RAG pipeline using KitOps, using @trychroma for the embedding database, #Llama 3 for our large language model (LLM), #SentenceTransformer for the embedding model, and @langchain for chunking. 👀 @tomaarsen codeproject.com/Articles/538…
2
3
68
#KitOps V0.2.5 has been released. This release includes two major improvements, 1/ Chunked blob uploads of ModelKits are now supported when available 2/ Kit CLI can remove ModelKits from remote repositories Check out the full release notes here: github.com/jozu-ai/kitops/re…
1
3
97
KitOps maintainer and CEO of @Jozu_AI, @BradMicklea joined @jeffbullas podcast last week to talk about AI and how to get your application to production in the fastest and safest way possible. #AIML #MLOps
Discover the hidden world of AI in this episode of the Jeff Bullas Show. Join us as we explore how businesses are leveraging AI to stay ahead, the unseen risks involved, and crucial strategies to protect your company's valuable secrets. jeffbullas.com/podcasts/secr…
1
3
137
We're excited to announce that Jozu Hub is now available in preview–Jozu Hub was purpose built to host ModelKits. Learn more here: dev.to/kitops/announcing-the…
1
2
67
Closing the enterprise AI divide–Lessons learnt from leading Amazon API Gateway by @BradMicklea #AWS #AIML #DevOps #Programming
1
2
125
Building AI? Check out letsbuild.ai - a community-driven platform dedicated to sharing resources, tools, and knowledge for AI enthusiasts and developers. We're excited to see #KitOps in the Model Packaging list. #AIML
2
20
KitOps V0.2.4 is now available. This release includes two major updates, making it over 90% faster to pack and unpack ModelKits. And new 𝚍𝚎𝚟 𝚜𝚝𝚊𝚛𝚝, 𝚍𝚎𝚟 𝚜𝚝𝚘𝚙, and 𝚍𝚎𝚟 𝚕𝚘𝚐 subcommands. See the full release notes here: github.com/jozu-ai/kitops/re…
1
2
166
Release v0.1.2 is out github.com/jozu-ai/kitops/re… Major fixes include: 1. Bug fix to ensure that the target folder is created during the unpack command 2. The ability to read a kitfile from stdin during the pack command
2
2
178
Building a #DevOps pipeline has become common knowledge among developers. But what if you had to build an #MLOps pipeline? No worries, we have you covered. In this post we breakdown the steps required to build your first MLOps pipeline using KitOps: jozu.com/blog/a-step-by-step…
1
2
56
Implementing AI/ML 🧠💻 doesn't have to be a hassle. We developed KitOps 🛠️ due to our belief in the potency of a standards-based approach ✅. Why? Because it ensures compatibility and uniformity 🔄 across an array of tools and platforms.
1
2
23
KitOps was added to @GitHub20k this morning 🚀 Check it out and give us an upvote♥️ gitlibrary.club/new/1
3
2
135
KitOps v0.3 is now available, this release includes: - The '-v' flag can be specified multiple times to increase verbosity (-vv, -vvv) - The ability to store generic parameters for Models in a kitfile - Kitfiles can now include documentation layers - More efficient local storage implementation that does not duplicate blobs - Support for client cert authentication - And, bug fixes and doc improvements For more information checkout the full release: github.com/jozu-ai/kitops/re…
2
2
347
Next we have @langchain#LangChain is a machine learning framework that enables ML engineers and software developers to build end-to-end LLM applications quickly. Its modular architecture allows them to easily mix and match its extensive suite of components to create custom LLM applications.
1
2
34
@portev33 a ModelKit is an OCI file type for packaging AI projects. You can learn more on kitops.ml
1
99
9. NannyML - (@TheRealNannyML) In machine learning projects, it is difficult to estimate whether improvement in model metrics will result in a positive change in business value. Furthermore, a model’s performance can change with time. NannyML is an open source #Python library that focuses on production monitoring and allows users to detect drifts (data and label), check data quality, estimate post-deployment model performance, and intelligently generate an alert if the drift is likely to impact model performance.
1
1
1
124
Replying to @attJOEah
we're trying to make this easier.
1
9
Next on the list is @hopsworks which is a perfect feature store for such architecture. It provides an end-to-end solution for managing ML feature lifecycle, from data ingestion and feature engineering to model training, deployment, and monitoring.
1
1
62
2. #Hydra (@MetaforDevs)- Hydra is a configuration management tool developed by Meta. It allows users to specify configurations via a file or the command line and supports hierarchical configurations. Hydra is extremely lightweight and easy to learn, making it ideal for beginners and experts. Furthermore, it allows users to run multiple jobs with a single command, which is ideal for the hyperparameter tuning of deep learning models.
1
1
41
Have you heard of @pachyderminc? It's is a data versioning and management platform that enables engineers to automate complex data transformations. It uses a data infrastructure that provides data lineage via a data-driven versioning pipeline.
1
1
1
175
We also have #DeepLake (formerly #ActiveloopHub) is an ML-specific database tool designed to act as a data lake for deep learning and a vector store for #RAG applications. Its primary purpose is accelerating model training by providing fast and efficient access to large-scale datasets, regardless of format or location.
1
1
1
132
Thanks for the shoutout @Prathkum 🥰🥰🥰
1
9
6. #Hyperopt - ML models involve numerous hyperparameters whose values can influence the model's overall performance. The only way of finding the best value for those parameters is to run the models with different sets of hyperparameter values, record the performance, and compare them. When done manually, this can be cumbersome. Hyperopt helps in automated hyperparameter optimization.
1
1
46
4. #Airflow - (@ApacheAirflow) An integral part of an ML project is data acquisition and data transformation into the required format. This involves creating ETL (extract, transform, load) pipelines and running them periodically. Airflow is an open source platform that helps engineers create and manage complex data pipelines. Furthermore, the support for #Python programming language makes it easy for ML teams to adopt Airflow.
1
1
41
@FeatureformML is a virtual feature store that streamlines data scientists' ability to manage and serve features for machine learning models. It acts as a "virtual" layer over existing data infrastructure like #Databricks and #Snowflake.
1
1
65
7. Weights and Biases - (@weights_biases ) is a tool for visualizing and tracking machine learning experiments. It supports major machine learning frameworks such as TensorFlow and PyTorch. Its key features include: It is a combination of DVC, CML, and Hyperopt. But, unlike those tools, Weights and Biases is only free for experimentation (single user) and academics. If you want to use it for a commercial product, you can host it on your own premises, if you have the compute resources, or use W&B’s cloud infrastructure.
1
30