Aug 5, 2024
Absolutely, thanks for your comment Chinmay. It goes to Data ingestions and I think I mentioned it in my story. sorry it wasn't clear. Data engineers MUST be able to tackle CDC and incremental updates. More in one of my previous stories here:
https://towardsdatascience.com/data-pipeline-design-patterns-100afa4b93e3