#schemaevolution

Jason Miles

@edudatasci.net

2 months ago

Gold as the Contract: Schema Evolution, Data Products, and Governance in Microsoft Fabric A schema change is rarely “just a schema change.” It’s the moment an upstream team’s perfectly reasonable adjustment becomes a downstream team’s broken report, confusing metric, or silent misinterpretation. And that’s why schema evolution has always been a source of anxiety: a schema isn’t simply structure—it’s an interface. In this post, I’ll do three things. First, I’ll ground why schema evolution has historically been such a persistent concern. Next, I’ll reframe Medallion with Gold as the published surface area of the data product, and Silver as an optional workshop layer where data is supplemented and transformed.

Stop treating #SchemaEvolution as an engineering failure. In a modern #MicrosoftFabric #Lakehouse, it’s an interface management problem: Bronze absorbs change, Silver optionally augments it, and Gold publishes the #DataProduct contract—governed, versioned, and consumable. #DataGovernance

Jason Miles

@edudatasci.net

4 months ago

Making Schema Change Boring: A Short History—and How Microsoft Fabric’s Medallion Lakehouse Bakes It In Schema changes have always been risky because a schema isn’t just columns—it’s the interface between data producers and data consumers. Historically, that interface was rigid, which made any change expensive. Modern lakehouse design solves the problem structurally: a Medallion architecture separates where variation is tolerated (Bronze) from where commitment is made (Silver) and relied upon (Gold). In Microsoft Fabric, those roles map cleanly to Lakehouse, Warehouse, and Power BI’s semantic layer, with governance and domain‑oriented (data‑product) design tying it all together.

Schema change isn’t a failure to control—it’s reality to choreograph. Use Fabric’s Medallion pattern to absorb change in Bronze, productize it in Silver, and deliver confidence in Gold. Governance and domains make it boring—and that’s the point. #SchemaEvolution #Lakehouse #DataProducts #DataGoverna

Particular Software

@particular.net

5 months ago

NServiceBus and Apache Avro • NServiceBus Using NServiceBus and Apache Avro.

Schemas are a great way to make versioning event-driven systems easier, and @ApacheAvro has the best schema evolution capabilities out there. And, you can use Avro with @NServiceBus too! Check out the samples:
https://docs.particular.net/shape-the-future/avro
#avro #schemaevolution #eda #serdes

Particular Software

@particular.net

5 months ago

NServiceBus and Apache Avro • NServiceBus Using NServiceBus and Apache Avro.

Schemas are a great way to make versioning event-driven systems easier, and ApacheAvro has the best schema evolution capabilities out there. And, you can use Avro with NServiceBus too! Check out the samples:
https://docs.particular.net/shape-the-future/avro
#avro #schemaevolution #eda #serdes

Particular Software

@particular.net

6 months ago

NServiceBus and Apache Avro • NServiceBus Using NServiceBus and Apache Avro.

Schemas are a great way to make versioning event-driven systems easier, and ApacheAvro has the best schema evolution capabilities out there. And, you can use Avro with NServiceBus too! Check out the samples:
https://docs.particular.net/shape-the-future/avro
#avro #schemaevolution #eda #serdes

Simon Späti 🏔️

@ssp.sh

1 year ago

The Evolution of Schema Change and database change management.

📖 New chapter: Evolution of Schema Change and database change management.

> #SchemaEvolution: Managing database changes while preserving integrity
> #NoSQL: Flexible schemas; speed over consistency
> #DataContracts: Producer-consumer agreements with automated validation

dedp.online/part-2/4-ce/...

Simon Späti 🏔️

@ssp.sh

1 year ago

I'm currently exploring #SchemaEvolution. Very early WIP, but do you see other pointers that relate to the history of schema evolution?

#databs

Hans-Peter Grahsl

@hpgrahsl.bsky.social

1 year ago

Schema Evolution in Change Data Capture Pipelines The importance of schema evolution for real-time data pipelines and how to modify data models without breaking compatibility with downstream consumers.

📝 #Blogged: I wrote this article to raise awareness of the importance of #SchemaEvolution for #realtime data pipelines built on top of #ChangeDataCapture. It discusses how to go about data model changes without breaking compatibility with downstream consumers.
www.decodable.co/blog/schema-...