Tom Cooper's Avatar

Tom Cooper

@tomcooper.dev

Senior Principal Engineer at IBM, Ex-Red Hat, Ex-Reddit, Ex-Twitter, Distributed Systems PhD, interested in all things stream processing, also cycling

208
Followers
151
Following
46
Posts
01.12.2023
Joined
Posts Following

Latest posts by Tom Cooper @tomcooper.dev

Kroxylicious Introduction
Kroxylicious Introduction YouTube video by Kroxylicious

Kroxylicious is the snappy open source proxy for Apache Kafkaยฎ.

See how it works in 3 minutes:
youtu.be/xLOjKScpJ3Q

#ApacheKafka #Kroxylicious #EventStreaming #DataEngineering

09.03.2026 02:58 ๐Ÿ‘ 1 ๐Ÿ” 2 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
the cast of Hackers (1995) posing in a series of adjacent phone booths

the cast of Hackers (1995) posing in a series of adjacent phone booths

Today is the 30th anniversary of Hackers

15.09.2025 23:56 ๐Ÿ‘ 3605 ๐Ÿ” 1045 ๐Ÿ’ฌ 72 ๐Ÿ“Œ 326
Phased upgrades of Strimzi managed Kafka fleets
Phased upgrades of Strimzi managed Kafka fleets YouTube video by Strimzi

A bit late, but the recording of my #StrimziCon2025 talk, Phased Upgrades of Strimzi Managed Kafka Fleets (hows that for a snappy title) is now up on Youtube: youtu.be/pEuYFFkQCUU?...

07.07.2025 10:33 ๐Ÿ‘ 2 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

It's my last day at Red Hat. Tomorrow I & the rest of middleware move to IBM. I wanted to work at RH since I heard about Fedora on the Linux Action Show (RIP) decades ago & I've loved developing in the open with RH. That'll continue after the move & after all, no one got fired for buying IBM...

30.06.2025 16:49 ๐Ÿ‘ 2 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

๐ŸŠ Kroxylicious Proposal: "Routing API"

This looks like a very useful addition to @kroxylicious.io (wire-protocol proxy for Apache Kafka): dynamic routing of requests to different upstream clusters, enabling use cases such as union clusters and client-based routing.

๐Ÿ‘‰ github.com/kroxylicious...

14.06.2025 09:26 ๐Ÿ‘ 10 ๐Ÿ” 3 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 1
Taming the Kafka Chaos: How Strimzi Helped Us Scale with Confidence Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.

Going from 1 to 13 PB/day ๐Ÿคฏ thanks to moving from a self-managed to a Strimzi-managed Kafka cluster! Great to hear these stories of how Strimzi not only made life easier for infra teams but improved the service they provide!

strimzi.io/blog/2025/06...

09.06.2025 13:50 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0

It was a really great talk, you covered a lot in 15mins! I am keen to work on an OSS library for embeddings wrapping Langchain4j so may ping you for opinions/reviews if you have time?

21.05.2025 20:35 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Luckily we have 2 people on the Strimzi team who literally wrote the book on Kafka Connect and Strimiz could do something (TBD) to help.

21.05.2025 16:11 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

Mostly about how everyone wants DR, try MM2, bounce off it and then end up with some very unique setups and trade offs. TL;DR MM2 (and connect more generally) is hard to get started with and has some sharp edges.

21.05.2025 16:09 ๐Ÿ‘ 2 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

"Process" not "Polymorphic" ๐Ÿคฆโ€โ™‚๏ธ

21.05.2025 14:15 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

Ending #Current25 with a talk on Polymorphic Table Functions in Flink๐Ÿค˜

21.05.2025 13:57 ๐Ÿ‘ 2 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

Great hallway track at #Current25, lots of ideas around Kafka connect, mirror maker, geo-replication, cluster balancing. Lots to mull over on the train home...

21.05.2025 13:14 ๐Ÿ‘ 3 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

"Vibe-coded database schemas" ๐Ÿ˜ฑ

21.05.2025 13:11 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

So nice to get a deep technical deep dive on a subject. They did a great job of explaining the migration process. If you want more detail we have blog posts:

21.05.2025 12:42 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

Woooo ๐ŸŽ‰ great to see Strimzi maintainers Kate Stanley and Paolo Patierno on stage at #Current25 talking about how Strimzi helps you migrate your K8s Kafka clusters from Zookeeper to KRaft

21.05.2025 12:04 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

After a very interesting lunch time at #Current25 speaking to folks from OpenAI about Mirror Maker and what we could do in Strimzi to make things easier, I'm watching the Flink AMA MC'd by @rmetzger.bsky.social

21.05.2025 11:27 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

My โšก talk choice at #Current25 is @hpgrahsl.bsky.social and vector stores in Flink

21.05.2025 10:01 ๐Ÿ‘ 1 ๐Ÿ” 1 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Ok so the main thrust is actually about removing Kafka from the mix & what you need to do in Iceberg to emulate topic compaction. A really interesting custom solution due to missing features in the Flink Iceberg connector. The headline, it works, it doesn't need Kafka but it's not as fast #Current25

21.05.2025 09:36 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

Onto something a bit more concrete now at #Current25. Improving CDC performance using Kafka, Flink and Iceberg

21.05.2025 09:06 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Kicking off day 2 of #Current25 by boarding the AI agent train๐Ÿค–๐Ÿš‚ with MongoDB ๐ŸŒฟ

21.05.2025 08:04 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image Post image

Wooop Open Source Datastack FTW! Pinterest moving from an in-house to OSS based architecture! #Current25

20.05.2025 16:47 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image Post image

Final talk of the day at #Current25, Async processing with Flink SQL

20.05.2025 16:31 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Great to hear that OpenAI have identified and fixed several issues with PyFlink and intend to push those fixes upstream ๐Ÿ‘๐Ÿ‘๐Ÿ‘

20.05.2025 15:50 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

It's PyFlink for all the things and OpenAI uses the upstream Flink Kubernetes Operator

20.05.2025 15:41 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Learning about how OpenAI built its stream processing system using Kafka and Flink at #Current25

20.05.2025 15:34 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

This was a very interesting talk full of intriguing ideas. However, there was a lot of "you _could_ do <complicated thing> and then do <even more complicated thing> based on that". I would love to see some real life demonstrations of all these agents linked together doing what we expect.

20.05.2025 15:11 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Post image

This talk has some really great examples of Flink's complex event processing (CEP) functionality in Flink SQL

20.05.2025 14:49 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Great to see a push to enhance upstream Flink SQL with addional AI orientated functions

20.05.2025 14:43 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Next up at #Current25 Flink ๐Ÿฟ๏ธ but with robots ๐Ÿค– (AI agents)!

20.05.2025 14:32 ๐Ÿ‘ 1 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

Next at #Current25 a panel discussion on the past, present and future of @kafkastreams.bsky.social. I love the background...the likeness is uncanny...

20.05.2025 13:17 ๐Ÿ‘ 6 ๐Ÿ” 2 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0