Weave AI Controllers: GitOps Automation for Large Language Models (LLMs)
Explore the future of Large Language Models (LLMs) with the new Weave AI Controllers. Simplify LLM adoption in enterprise environments through GitOps automation, addressing challenges like data privacy and proprietary data protection. Discover how Weave AI Controllers streamline operations, enabling self-service platforms for developers and facilitating secure fine-tuning for enterprise-scale efficiency.
It’s with great excitement that we announce the release of the Weave AI controllers for Large Language Models (LLMs). Weave AI controllers ease the adoption of open-sourced LLMs like Llama-2, Mistral, Zephyr, and Falcon in enterprise environments through GitOps automation. We made it simple and efficient for Machine Learning teams to deploy, manage and fine-tune LLMs on any Kubernetes infrastructure while ensuring strong security and governance. Recent CNCF surveys show GitOps is the standard operating model for Kubernetes based workloads.
LLMs are on the rise but so are complexities in operating them
The usage of LLMs has grown dramatically over the past several years and so has their rate of adoption. According to this survey, nearly one in ten (8.3%) machine learning teams have already deployed an LLM application into production and nearly half (43.3%) have production deployment plans within the next 12 months. However data privacy and the need to protect proprietary data are the largest roadblocks for production deployment. Many organizations are struggling with manual, ad-hoc methods of deploying downloaded LLMs that can lead to security risks and lack of governance and compliance. Versioning and updates as well as integration into existing infrastructure are also hurdles that need to be overcome.
Weaveworks addresses these challenges with GitOps workflows, Flux-based AI Controllers, and a signing and verification process that enhances security and compliance even in regulated industries. The Weave AI Controllers will be shipped beginning December 2023 with our standard subscriptions in Weave GitOps Assured and Enterprise.
Streamline ops and free up development time
Weave AI controllers were designed to address two main use cases:
- Enabling a self-service platform of AI models, tools and applications for developers, and
- Facilitating fine-tuning of models with sensitive data for enterprise-grade efficiency, security, and reliability.
Many ML teams have been exploring these models and their capabilities in development or on small scale production clusters, with Weave AI Controllers teams can now move into enterprise scale with the necessary security guardrails and deploy to production quickly.
A Kubernetes based infrastructure and deployment pipeline using CRDs, YAMLs and GitOps can easily remedy most deployment challenges while provisioning monitoring and rollbacks. Weave GitOps and our AI controllers are leveraging cloud native technologies and the declarative management approach to build automated and streamlined workflows on prem, hybrid or in the cloud. We want data scientists to focus on the application and stop worrying about infrastructure tasks.
Why choose Weave AI controllers
- Security, Governance & Compliance: Weaveworks enhances security and compliance by using the signing and verification process offered by Flux OCI and Source Controller. This process ensures that deployed models are free of vulnerabilities or malicious modifications, and adheres to governance and regulatory compliance standards.
- Enhance and not replace: We designed Weave AI Controllers to seamlessly integrate with existing systems, using current Kubernetes clusters to manage LLM workloads. They work on Red Hat OpenShift out-of-the-box, facilitating deployments on various backends without disrupting workflows.
- GitOps for Productivity & Collaboration: Weaveworks has built an incredible ecosystem around Flux and Kubernetes to enable GitOps automation. Our Weave AI controllers are an extension that allow you to build on tools you already know. You don’t have to introduce changes to your workflows but can reap the benefits of consistent ops practices, observability and additional security.
Weave AI Controllers are an out of the box extension for FluxCD and are shipped with our commercial subscriptions for Weave GitOps Assured and Enterprise. Weave GitOps Assured is our comprehensive support for the OSS cloud native toolkit including Flux, Flagger, Flamingo, Terraform controller and more. If you are already operating on an enterprise scale, Weave GitOps Enterprise is the trusted Kubernetes automation for platform and application teams.
To see a demo of the Weave AI controller live, please join us for our upcoming office hours on November 29 at 10am PT - sign up today.
Nov 29: GitOps Automation for Large Language Models
See a demo of our latest Weave AI controllers for FluxCD.
