Factor House

Lots happening in October, from community sessions to a trip to Chicago. Swipe through to see where to find the Factor House team this month 🌐 Oct 6: Migrating to open source Kafka, with Chad Harris and Justin George from NetApp Instaclustr 🌐 Oct 14: Beyond IAM: governing Amazon MSK for teams and AI agents, with Chad Harris 🌐 Oct 28: Virtual Kafka User Group, Americas, with a lightning talk from Yaroslav Tkachenko (formerly Activision, Shopify) 🌐 Oct 28: Virtual Kafka User Group, Asia Pacific, with a lightning talk from Gaurav Bhatt (Sportsbet) 🇺🇸 Oct 29: Databricks Data + AI World Tour, Chicago Then on Nov 4-5 we're heading to 🇺🇸 Current San Francisco at Moscone Center. Come and find us at booth 403. Links to everything are in the comments

Operating Kafka on Amazon MSK with more than one team on the cluster? Join Chad Harris on October 14, he's an expert in this space, and has plenty of experience to share, don't miss it! 👇

Join us on October 6 to learn about cutting your Kafka infra costs, with experts Chad Harris and Justin George, NetApp Instaclustr. They'll walk through comparisons and the trade-offs to weigh when considering open source Kafka.

September 2026: Cutting Kafka costs through cluster consolidation, a live preview of agentic Kafka tooling, governing Amazon MSK for teams and AI agents, and how Netflix runs Apache Flink in production

Deploy MirrorMaker 2 on Kafka Connect the obvious way, and your target cluster fills with a second copy of every record from the beginning of time. That's because replication progress doesn't migrate. A new source connector starts at offset zero. Chad Harris has written a practical guide to moving MM2 off connect-mirror-maker.sh and onto Kafka Connect, covering: → How to seed offsets so nothing is re-replicated → Why consumer offset translation is off by default → The cutover order that keeps things uneventful → When you shouldn't bother moving at all Link in comments.

Excited to have kicked off our first New at Factor House session this morning. Got to show off the new CLI and agent skills for Kafka operations live, plus a bit more. Missed it? We're running it again next week for Europe. Link in comments.

Join us on September 29 for a look at what's new across Factor House's products this month, with Chad Harris and Derek Troy-West. They'll walk through live demos of recent capabilities and the thinking behind them, plus a preview of what's coming next - bring your questions, the floor's open throughout.

Thanks to Stefan Baer, Senior Key Expert in Data Integration at Siemens, for a great lightning talk at last week's Kafka User Group - Europe. Stefan walked through how Siemens tackled schema quality across multi-producer Kafka topics, where schemas were "agreed" between teams but never validated, so one producer's mistake flowed through to every consumer. A few takeaways: → Using Claude Code to rebuild a working schema from real production messages, surfacing drift the existing schema had missed → Balancing validator strictness: too strict rejects everything, too loose accepts almost anything → A staged rollout, from consumer-side validation, to a gatekeeper at the merge point, to producer-side validation Want to join the next Kafka User Group? Registration link in the comments

Dead letter queues in Kafka usually mean bolting on a stream processor or an external retry service. Chad Harris has put together a walkthrough of a consumer-side approach that skips both, using nothing beyond Kafka itself: → Each consumer group gets its own retry and DLQ topics → A failed message only replays to the consumer group that failed, not every group subscribed to the topic → No new infrastructure to run or maintain If your team has dealt with fan-out problems from a single shared DLQ, this is worth a read. Link in comments.