Did you know you can purchase #deltalake the definitive guide from reputable non-monopolistic book sellers?
You can also pester some of the other authors and I next week in Mountain View, CA: https://lu.ma/okxq0bt1
Recent searches
Search options
#deltalake
0 posts0 participants0 posts today
Just caught up with the recent Delta Lake webinar,
> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust
Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. #ApacheAirflow tasks
youtu.be- YouTubeEnjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.
Revolutionizing Data Management: Mooncake-Labs Unveils pg_mooncake v0.1.0
In a significant leap for data processing, Mooncake-Labs has launched pg_mooncake v0.1.0, introducing robust features that enhance PostgreSQL's capabilities. This release empowers users to seamlessly ...
Revolutionizing Data Management: Mooncake-Labs Unveils pg_mooncake v0.1.0
In a significant leap for data processing, Mooncake-Labs has launched pg_mooncake v0.1.0, introducing robust features that enhance PostgreSQL's capabilities. This release empowers users to seamlessly ...
I hope you'll be able to join the #deltalake AMA today at 9am PST with the authors of Delta Lake The Definitive Guide 
https://www.linkedin.com/events/asktheauthors7257759127987261441/
I am so so excited to have a physical copy of #deltalake "The Definitive Guide"
I had some talks earlier this year about #deltalake and #rustlang
I also did so with one hand...because I broke my wrist at the beginning of the conference 
brokenco.deRemember FastCGI?
a moderately technical blog
If you use #deltalake from Python or Rust, I sure would appreciate it if your company sponsored my work!
GitHubSponsor @rtyler on GitHub Sponsors👋 I am R Tyler Croy, hobby hacker and contributor. Once upon a time I helped build the Jenkins project and community, but more recently I have been focusing on Rust-based network services for autom...
If you use #deltalake for #python, you can now sponsor one of it's main contributors Ion!
GitHubion-elgreco - OverviewData Engineer. ion-elgreco has 18 repositories available. Follow their code on GitHub.
We are excited to announce the release of python-v0.19.0! 
Highlights include:
CDF support in write_deltalake, delete, and merge operation Expired logs cleanup during post-commit. Can be disabled with 𝚍𝚎𝚕𝚝𝚊.𝚎𝚗𝚊𝚋𝚕𝚎𝙴𝚡𝚙𝚒𝚛𝚎𝚍𝙻𝚘𝚐𝙲𝚕𝚎𝚊𝚗𝚞𝚙 = 𝚏𝚊𝚕𝚜𝚎 Improved MERGE performance by using predicate non-partition columns min/max for prefiltering 𝙰𝙳𝙳 𝚌𝚘𝚕𝚞𝚖𝚗 option Speed up log parsing
View the release notes: https://github.com/delta-io/delta-rs/releases/tag/python-v0.19.0
The #deltalake Python and Rust bindings have change data capture (CDC) read and write support in the works right now.
*Very* exciting stuff happening between now and Data and AI Summit in June!
Finally hitting a groove with this #deltalake code at ... almost 23:00 local time 
I had fun sharing the latest and greatest in the #deltalake #rustlang project.
Our next release 0.17 will include some really important enhancements which will go live next week!
Learn more here:
https://youtu.be/300wnXIwVP0?t=327
TOMORROW, January 31st at 9AM PT, join us for #DeltaLake Deep Dive: Rust Crate with @rtyler, Delta-rs Maintainer and founder of Buoyant Data! 
Learn about the upcoming enhancement in Rust's crate capabilities.
Say goodbye to the old DynamoDB locking mechanism and welcome the innovative Delta/Spark compatible concurrency support - the S3DynamoDbLogStore.
RSVP on LinkedIn 
https://hubs.la/Q02ghtvC0
RSVP on Meetup https://www.meetup.com/data-ai-online/events/298469304/
I am thrilled that I am able to hire another full time Senior Data Engineer in North America for my organization at Scribd.
At a high level my team:
* created the #deltalake Rust bindings, kafka-delta-ingest, and a number of other Rust based data tools.
* has presented multiple times at Data and AI Summit
* works at the leading edge of the Databricks platform
* is remote-first, spanning 3-5 continents depending on the time of year 
(we write a lot of stuff down!)
https://jobs.lever.co/scribd/80fcabda-99a9-4b79-b125-e1687baee19d
jobs.lever.coScribd - Senior Data EngineerWhat You'll Do: You will play a pivotal role in managing and optimizing our data stream of analytics events. Your responsibilities will include designing and implementing robust data pipelines and transformations, ensuring data integrity through automated quality monitoring. We’re looking for someone who is not only skilled in data engineering but also understands the high stakes of working with business-critical data. Your expertise will directly contribute to the efficiency and reliability of our financial operations, making it essential for the success of our partners and our business. Based on the project, this might involve cross-functional work with the Data Science or Content Engineering teams to troubleshoot, process, or optimize our business-critical pipelines, or working with Core Platform to implement better processing jobs for scaling our consumption of streaming data sets. Almost everything you would be working on would be to increase the "customer satisfaction" for inte
In two weeks, @rtyler, Delta-rs Maintainer and founder of Buoyant Data, will talk about the upcoming enhancement in #Rust's crate capabilities!
Say goodbye to the old DynamoDB locking mechanism and welcome the innovative Delta/Spark compatible concurrency support - the S3DynamoDbLogStore. 
Bring your questions and get ready for an engaging Delta Lake Deep Dive on Wednesday, Jan 31st.
RSVP: https://www.meetup.com/data-ai-online/events/298469304/
MeetupDelta Lake Deep Dive: Rust Crate, Wed, Jan 31, 2024, 9:00 AM | MeetupLearn about the upcoming enhancement in Rust's crate capabilities in our upcoming Delta Lake Deep Dive with R. Tyler Croy, Delta-rs Maintainer and founder of Buoyant Data!
Learn about the upcoming enhancement in #Rust's crate capabilities in our upcoming Delta Lake Deep Dive with @rtyler, Delta-rs Maintainer and founder of Buoyant Data! 
Say goodbye to the old DynamoDB locking mechanism and welcome the innovative Delta/Spark compatible concurrency support - the S3DynamoDbLogStore. This enhancement is a game-changer, thanks to the awesome efforts of Thomas Peiselt.
RSVP: https://www.meetup.com/data-ai-online/events/298469304/
How does Delta Lake quickly compute state? Here’s a quick primer on how #checkpoint files are used to speed up the process.
https://dennyglee.com/2024/01/09/computing-delta-lake-state-quickly-with-checkpoint-files/
Denny Lee · Computing Delta Lake State Quickly with Checkpoint FilesTo avoid the “small-file” problem, Delta Lake creates a checkpoint file in Parquet format as a shortcut to speed up state extraction.
I am looking to line up more #deltalake contract work for myself and a few others, I'm particularly interested in finding a company wanting to take a chance at deploying Ballista
I submitted two #deltalake related talks to Data and AI Summit, hopefully they're both accepted. If you want to learn how to solve data engineering processes with #rustlang, you'll want to come to my talks :)