Marvelous MLOps

Maria and I are back with our live cohorts, and we’re excited to meet both new and returning students in our previous MLOps course. This time we’re going deeper into LLMOps with Databricks and it's starting in just 2 days! Our MLOps course has helped hundreds of practitioners learn how to take ML models from prototype to production with confidence. Many of the same principles apply to LLM applications but LLMOps is more than prompt engineering. It’s about applying proven MLOps practices to LLM systems: reproducibility, reliable deployment, monitoring, and managing cost, performance, and safety in production. In the course we’ll cover: 🔹Evaluating and tracing LLM apps with MLflow Tracing & Evaluation 🔹Logging agents with MLflow and registering them in Unity Catalog 🔹Databricks Vector Search 🔹Deploying agents with Mosaic AI Model Serving 🔹Managed Databricks MCP and custom applications 🔹Databricks Asset Bundles 🔹Applying CI/CD and DevOps patterns to LLM development Looking forward to seeing many of you there 🚀 https://lnkd.in/eVkBinJb

We’ve received a lot of positive feedback on our MLOps with Databricks course on Maven. And yet, we decided to close it last year. While traditional ML still delivers most of the real business value, the market has clearly shifted toward Agentic AI. But MLOps didn’t disappear. If anything, it became more advanced, more complex, and more critical than ever. That’s where we kept seeing the same gap. Most courses teach you how to build agents, almost none teach you how to operate them in production. So Başak and I decided to focus entirely on that gap. End-to-end LLMOps on Databricks, which is moving insanely fast in this space. There’s currently no other course that focuses on Databricks and goes this deep, this practically, and this up-to-date on: - Building agents on Databricks - Operating them in real production environments - Managing the full lifecycle around them Bonus: you’ll also get full access to our MLOps materials. We’re not planning to run this course again anytime soon, so grab this opportunity! P.S. A few students ran into payment issues earlier, so we’ll have a 20% discount open until the end of January: https://lnkd.in/efAyrwE4

Databricks Model Serving is one of the fastest-growing features on Databricks right now. And honestly, the list of reasons not to deploy on Databricks keeps shrinking. That said, I would still not deploy on Databricks if: - I need to connect to databases or systems outside Databricks that aren’t exposed via public IP (I hope yours are not!) - I need to call Foundation Model APIs that aren’t reachable via public IP - I need heavy user-level A/B testing and total in-memory state exceeds 4GB (hard cap from Databricks) - I need very fine-grained autoscaling behavior. Scaling on Databricks is based on estimated concurrency (RPS × model predict time). It will still autoscale when CPU is underused, so if you want perfect utilization tuning, Databricks may not be your ideal setup - I need massive scale: workspace-level RPS limits apply. You’re not serving 200k RPS from Databricks (but also… most companies aren’t either) In those cases, I’d still log models in MLflow, register in Unity Catalog, and deploy on Kubernetes instead. For Kubernetes-native deployments, KServe (free) and Seldon (quite expensive) are out-of-the-box options. You can also build your own vanilla deployment if you want full control. What you do get from Databricks Model Serving (and not easily from other options): - Inference tables - AI Gateway - Built-in usage monitoring - Tight integration with the Databricks ecosystem And no matter what you serve on, you should use MLflow Tracing for your AI applications. MLflow traces can sync into Delta tables (currently ~15 min latency, but should get near-real-time soon!) Thanks to Denny Lee, Kerry Chang, Jeroen Meulemans, Rob Jelsma, and Bryan Qiu for great discussions! Excited to see how MLflow and model serving on Databricks is improving!

As an ML/AI engineer, I depend heavily on infrastructure, but actually deploying that infrastructure is rarely the fun part. I’ve fought with Terraform enough to know it’s not for me: clunky, slow to iterate, and painful to debug. I prefer Python-first tools like the AWS CDK, but that doesn’t help when I need the same experience on Azure. Pulumi has been on my “should really try this” list for a while, and when Pulumi released Pulumi Neo, I finally dove in. Neo gives you the Pulumi experience with basically zero effort. It’s a chat interface inside Pulumi: describe what you want, and it builds + deploys the infra. I deployed a DynamoDB table without writing a single line of code. My workflow: - create an IAM user - add credentials to Pulumi, share account + region - create a Pulumi GitHub app - create a Pulumi secret and store it in my GitHub repo - tell Neo: “spin up a dynamodb table” That’s it. Fewer iterations than I expected, and I ended up with a repo containing working infra code and deployment pipelines. And sure, this was a toy example. Neo can go way bigger: updating Kubernetes clusters, managing complex configs, deploying full multi-service setups. Pulumi + Pulumi Neo is absolutely worth checking out. Try Pulumi Neo (it is free, no credit card required!): https://lnkd.in/eRenmyin Learn more about Pulumi Neo: https://lnkd.in/ex7zuSiX -------------------------------------------------------- I partnered with Pulumi for this post, but everything here reflects my own experience.

Most ML engineers learn how to train a model. Very few learn how to ship one. That’s where this course comes in. The MLOps with Databricks - Free Edition is built for anyone who wants to go beyond notebooks and understand what real production ML actually looks like. No fluff. No long theory detours. Just practical, end-to-end MLOps on Databricks. Inside the 10-lecture series, you’ll learn how to: 🔧 Build reproducible workflows 📊 Track experiments & manage models with MLflow ⚙️ Design model-serving architectures 🚀 Deploy endpoints the right way 🔁 Add CI/CD to your ML pipelines 📈 Monitor models in the Lakehouse It uses the same tools real teams run in production, Databricks + MLflow, so everything you learn maps directly to real-world work. 🎥 by Maria Vechtomova & Başak Tuğçe Eskili p.s. Your future ML career = how well you understand production pipelines. ( I will put the playlist in the comments) ♻️ Repost to help others learn from this course too. Follow Mayank for more insights on AI/ML :)

Here is your starter pack for uv, the modern Python environment manager. ▶️ One command handles everything: uv sync ▶️ Works across Mac, Linux, Windows ▶️ Sets up your venv, dependencies, and CLI app in minutes See the full guide in the latest article! Written by Boldizsár Palotás

❗Warning for anyone using #Databricks Online Tables / Feature Serving Databricks is decommissioning online tables. Going forward, the recommended approach is to use the Databricks Online Feature Store (which uses Lakebase) and publish feature tables to an online store. This change has significant cost implications. We noticed it firsthand while teaching our last MLOps with Databricks cohort: same lab materials, smallest possible online-store instances, yet our “Database” costs spiked. Originally, we let every student create their own online store. In hindsight, it’s more cost-efficient to reuse online stores whenever possible rather than creating new ones per project. If you’re planning labs, demos, or production workloads, keep this in mind!