Also got to attend a workshop hosted by Runway folks earlier this week here in Toronto - I really loved the product!
09.05.2025 19:50
👍 1
🔁 0
💬 0
📌 0
Also got to attend a workshop hosted by Runway folks earlier this week here in Toronto - I really loved the product!
Shivanshu Gupta is a Data Engineer at Chainalysis, where they run Dagster to orchestrate and maintain complex data pipelines.
🔍 In today's data landscape, engineers like Shivanshu need tools that reduce cognitive load while handling increasingly complex pipelines.
Used Cursor briefly but have recently been pretty happy with GH Copilot + VS Code Insiders which supports agent mode
8/10 times the latter
Pretty cool - ty for sharing!
This is great, will there be a direct link to the talk so I can share with my team?
right*
You are write in that you still have to call `op` every time, and that can be annoying. I’d like to work out a solution that can set env vars for the current shell session using `op` so you don’t have to call it every time.
The CLI has become a part of my dev workflow. I wrote about it here, but the gist is: you can have your environment vars populated dynamically by 1Password using a bash alias and a .env file. www.sgupta.xyz/posts/secret...
Yep. Any company claiming AI is ready to replace humans who provide value that goes far beyond writing code doesn’t know what it’s talking about
Makes sense when you put into perspective the $ constraints + the relative nascency of the Iceberg SDK ecosystem. Appreciate the insight, @benesch.bsky.social.
S3 (Iceberg) Tables is everything I dreamt of, and more. I blogged some long-form thoughts: meltware.com/2024/12/04/s...
I think we're about to see an explosion of data tools (@materialize.com, @clickhouse.com, @duckdb.org, et al.) learn to write Iceberg tables via S3 table buckets.
#databs
Great read! It sounds like the absence of table compaction feature in the Python and Rust libraries is a blocker to write support, curious to hear your thoughts on why vendors have not taken on a more active role in contributing towards this feature in these libraries. Thanks!
Awesome. Will DM you. Huge fan of the product already btw, and Rill fills the operational/tactial, low-latency analytics gap in BI in a way that no other tool does, kudos to you and the team!
Anyone here try to self-host Rill (@rilldata.com) or is that not a thing? $250/mo to deploy on Rill Cloud feels prohibitive for most hobby projects... #databs
Definitely feel this way with Pydantic/type hints for most production-bound Python code I’ve written recently!
1
epochconverter.com still balling
this is the only good WebApp out there, don't at me
I've been seeing posts from people coming to #dataBS and getting overwhelmed and worrying they can't contribute.
I get it. I still feel that sometimes.
But I'm trying to use my impostor syndrome as a strength. It's permission to accept that we'll never know everything.
So join in the discussion!
Anyone here manage to attach a Databricks-hosted Unity Catalog to DuckDB? github.com/duckdb/uc_ca...
Love that #DuckDB & #R2 are powering quick & easy access to open datasets. How long until we get the entire library of BQ public datasets but without the GCP fluff? #DataBS
Custom Bluesky handle off my domain, neat
I open my Bluesky feed and it is full of really cool people saying really smart things and being really passionate about really interesting stuff.
I like this version of social media, and the world.
Thanks, folks.
I underestimated how far you can get with prompt engineering. Every time I thought I needed to bite the fine-tuning bullet, what I really needed was better prompts
Going from CSV > #DuckDB > Delta Lake in S3 is a breeze. Only thing that's missing is write support to Delta Lake tables natively in SQL (vs having to go through Python). And DuckDB Google Sheets integration makes me want to use it for everything
Starting to really love DuckDB
#databs