Wednesday, November 9th, was a big day for us Craties and our Community: we had our very first CrateDB Community Day!
With attendees from all over the globe, we shared knowledge, stories, and even a few jokes throughout 6 great talks 🙌
CrateDB Community Day was hosted by 2 Craties from Crate.io's Customer Engineering team: Marija Selaković, Developer Advocate, and Rafaela Sant'ana, Developer Advocate Intern. From the first minute, they did a great job welcoming us all 😊
Let's take a look at some of the highlights of each talk:
Our journey with CrateDB, by Christian Lutz, Founder and Director of the Board at Crate.io💡
Christian shared CrateDB's journey with us and how it all started with meet-ups with pizza and beer. Like all entrepreneurs with very early products, Christian shared the challenges of building a presence in a very competitive market.
Thanks to CrateDB's distinct characteristics that they had in mind, it caught the eye of some developers and one of those developers wrote a post "big data in real-time with sequel". This statement created a lot of buzz around our product and accelerated the release plan:
"CrateDB can simply be deployed Cross operating system on cheap commodity hardware. It will not die and it can handle billions of records with sub-second latency. This is how we started with the whole thing"
Today, Christian thinks there are excellent reasons to make tactical decisions, you have specific databases for specific use cases and tasks and now it's just a mouse click away for you to scale, it's just very easy.
Our attendees had the opportunity to ask Christian some questions, one of the many insights he shared was the most significant learning from building the product and company:
"There are a ton of things the team and I have learned. Total focus on customer value and benefits, on generating early revenues, and to build and rely on a stellar team"
Analyzing and Visualizing Twitter Conversations, Candelario A. Gutierrez, M.Sc. in Computing Science 💻
Candelario kindly shared with all of us how they leverage CrateDB into their platform for analyzing social-platform data, including the objective of his work, some essential concepts and motivation, the components of the data platform, and a review of an energy case study.
"We wanted to explore what people had to say about different energy types and/or what type of conversations were happening under a social platform"
According to Candelario, analyzing this type of data can be a really big challenge, because the social platform that has been continually increasing in volume and velocity, the complexity of big-data software frameworks, all of this and more increases the learning curve time of the developers.
"We believe that there are a wide range of studies that can get benefit from such a system that we propose and can give support during the analysis of the data pipeline in order to help with its retrieval processing, storage, and visualization."
Candelario presented a detailed content analysis, the case study findings, and really interesting conclusions, including two bottlenecks within the processing pipeline, discovered different events to understand people's points of view and that no matter the tables needed, they experienced a really good writing execution time:
"CrateDB was a good fit to handle the requirements for this specific case study and for this type of data, with really low latency"
While answering a question about CrateDB's most useful feature for him, he shared that the user experience, exploring the dataset before the creation of visualization, all databases organized on one side, and doing his own queries, was super helpful.
"With CrateDB, we have a really awesome community that helps us!"
How Bitmovin uses CrateDB to monitor the biggest live video events, Daniel Hölbling-Inzko Director of Engineering – Analytics, Bitmovin ⏯️
As Daniel pointed out, Bitmovin has been working with Crate.io since 2016, which gave them extensive knowledge of how to run it to their specific needs! In this talk, Daniel shared with us how Bitmovin uses CrateDB to monitor live video events and which features they are using to address their monitoring and scalability challenges.
After a wonderful introduction about Bitmovin, how they collect data from video players, insights about video metadata, process and stores data, and much more valuable insights about their experience, Daniel shared their learnings while using CrateDB:
"CrateDB stores lots and lots of data. Each playback session generates around 100 rows of data, for us, 1 day easily represents 1TB of data"
Daniel also highlighted that data needs to be real-time, which is essential in these live events, as the operations team depends on this data to make decisions.
"Getting Crate's support was a key enabler for us, having their team support is very helpful"
Introduction to Using Apache Airflow with CrateDB, Tamara Janina Fingerlin, Developer Advocate, Astronomer 👩🏻💻
In this insightful talk, Tamara's main goal was to show our attendees all the necessary steps to spin up an airflow instance locally and run a SQL query within CrateDB instance.
Tamara started with a short introduction of what is Apache Airflow: an open-source tool for programmatically offering, scheduling, and monitoring your data pipelines, a data orchestrator, that it's highly flexible.
In this talk the attendees had the opportunity to learn how to navigate their Airflow project and some of the benefits like the fact that Airflow is agnostic, integrating your whole data ecosystem, its flexible scheduling, infinitely scalable Kubernetes, and more!
Tamara shared her thoughts on the first edition of CrateDB Community Day:
"It was very interesting, learning so much about the in-depth workings of a database solution. It's exciting and very different than what I usually hear."
Introduction to CrateDB Cloud, Roman Meingassner, Senior Product Manager, and Romanas Sonkinas, Lead Engineer of CrateDB Cloud, Crate.io ✌🏻
Thanks to this dynamic duo, we all learned more about CrateDB Cloud in a demo environment. CrateDB Cloud is a fully-managed database-as-a-service, that enables data insights at scale. Available on Azure, AWS, and soon on Google Cloud.
We highlight the ease of configuration, deployment, scalability, and management!
In their very insightful (and fun) talk, our attendees had the opportunity to see in real-time how to start and scale the new cluster, implement logical replication, run data ingestion, and understand the actual costs behind CrateDB Cloud.
"Upgrades are easy, scaling is super easy, as well as expanding and storage and we firmly believe this is the future!"
Roman and Romanas also shared during the Q&A that backups work automatically once an hour, and are kept for 2 weeks. Regarding the pre-credit, they explained that:
"If you create an account you get free credits ($200) and it's valid for 30 days, it will not start to charge until you spend your credit."
Make use of CrateDB's performance potential, Marios Trivyzas, Senior Software Engineer, Crate.io 📈
Marios, with his extensive experience with CrateDB, shared his knowledge with all participants on sharding & partitioning choices, along with correct data modeling, and how to monitor your cluster to avoid unpleasant surprises.
Staring with scale out as your data grows, Marios shares valuable insights like "too many shards use cluster resources, too few, you don't use the full performance capacity of your cluster" and the ideal scenario to guarantee your cluster stays healthy at all times!
We got to know some key factors which will allow to deploy CrateDB to the cloud and achieve the desired performance.
One interesting question came up: what makes CrateDB different from other Lucene-based solutions?
"In CrateDB you have build from scratch distributed SQL execution engines. It's not many systems that have this SQL plugin on top (...) The administration of your cluster can be done through pure SQL, you don't need to learn any new language."
After the talks ☕
After the Q&A, Christian shared his thoughts on "What's next for CrateDB"
"In Crate.io we have different streams at the same time: add features that the user asks for, improve, the backlog of a lot of ideas and wishes from customers and our team, and finally, what happens in the market and how to react. The new version of CrateDB Edge is of the bigger milestones"
Marios added:
"Generally we try to improve to facilitate the tool and work smoothly with CrateDB, improve the user experience, constant improvements in SQL, try to keep up with our customer's needs."
Would you like to know more regarding the first edition of Community Day? Watch the recording to get all the details!
Give us a ⭐ on GitHub and keep up with the news in our social media channels LinkedIn, Twitter, Facebook 👈