Swapnik's Avatar

Swapnik

@swapnikc

Data Engineer by skill. Engineering Manger by profession. Interests in AI, Data, Sustainable Engineering

45
Followers
389
Following
7
Posts
01.01.2025
Joined
Posts Following

Latest posts by Swapnik @swapnikc

Is this for real?

05.07.2025 16:01 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0
Post image

I ask chat gpt 4.5 (research preview) if I am speaking with it and it says this

#gpt4.5 #chatgpt

06.03.2025 17:13 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0

#dataanalytics #datawarehousing #dataengineering #google

21.02.2025 13:25 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0

Think of the api layer like a gateway to distribute the load to multiple servers

21.02.2025 13:24 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0

Our use case was primarily a duckdb to delta connect and agg on top of it. We have used a 128gb machine and wrote a custom api layer for taking queries.

21.02.2025 13:23 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 0 ๐Ÿ“Œ 0
Preview
Why Google Napa Chooses LSM-Trees Over B-Trees: A Deep Dive Because Random Disk I/O is a Luxury

Check out my take on BTrees vs LSM trees and how google napa used these to the core for petabyte scale analytics.

open.substack.com/pub/datashot...

21.02.2025 13:22 ๐Ÿ‘ 0 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0

DuckDB is at best a middleware. Doesn't scale like DWH. Our recent tests with remote delta reads have proved this.

01.01.2025 07:15 ๐Ÿ‘ 2 ๐Ÿ” 0 ๐Ÿ’ฌ 1 ๐Ÿ“Œ 0