An implementation of Airbyte Protocol in Rust

Last year, I worked on integrating Airbyte and Apache Hudi. They are Open Source projects in Data Engineering and DataLake/Lakehouse space.

Read about that in the blog here

Since then, both projects moved at a fast clip. I could not keep up with rebasing changes from Airbyte as the design I had was complex enough to touch many parts.

The main idea was to bring an intuitive UX to Apache Hudi from my experience after spending a great deal of time operationalizing it at my last job. But at the same time, Onehouse was born. It is founded by Apache Hudi core team members.

So it is kind of redundant for me to build that awesome UX for Hudi anymore.

Meanwhile, life is taking me for a wild ride and I took a break for the last few months. But kept on thinking about how to leverage all that effort I put into understanding and re-architecting the Airbyte codebase. So decided to give it a try in a different way as I got some time to finally focus. Thanks to the parents.

Felt like it is taking good shape, but not completely done. Sharing it for early feedback and failing fast. Also to find any like-minded collaborators.

The Code is here,

A hands-on tutorial blog post will follow shortly.

Regards,
Harsha Teja Kanna