The Guide for Time Series Data Projects is out.

Download now
Skip to content
Get Started
Get Started
Customers

CrateDB provides the backbone of Bitmovin's real-time video analytics platform

By leveraging the CrateDB database, Bitmovin, a leading provider of video streaming solutions, can control its massive data flow while reducing costs.

About Bitmovin

Founded in 2013, Bitmovin is a leading video streaming company that built the world’s first commercial adaptive streaming player and deployed the first software-defined encoding service that runs on any cloud platform.

Its portfolio includes services for scalable video encoding, the Bitmovin Player for playing videos on all platforms and devices, and a complete end-to-end streaming solution with Bitmovin Streams. The company offers a product for monitoring and analyzing video streaming data, which is used by the biggest media companies around the world to deliver high-quality video experiences to end users.

This function records certain metadata every time a video is played. This provides streaming providers with information on who watched the video, when it was started or stopped, and on which device it is running, among other things. Bitmovin produces billions and billions of lines of data, which streaming providers can use to monitor and analyze their customers’ user video experience.

cr-customers-bitmovin-video

Challenge

Since the advent of video streaming, a lot has happened in this area. On the one hand, the quality of the streams is constantly increasing and, on the other hand, the number of companies and service providers offering video-on-demand is also growing. This situation increases the pressure on individual providers not only to offer their users the best content, but also to deliver customized content of the highest quality.

To meet these demands, streaming providers need to know their users, which is the only way they can optimize their offerings. This insight was the starting point for Bitmovin's video analytics solution.

Having already been very successful with its video encoding services and its streaming solutions including video players, Bitmovin started working on its new product, which aims to offer users – regardless of their video player – capacities for the collection and analysis of large and fast-moving data sets in real time.

However, the broadcast of large events, such as the World Cup, provide extremely high data throughput that traditional databases either cannot handle or can only handle at a massive cost. Moreover, Bitmovin did not want to make this solution available only to a small selection of hand-picked users but wanted to roll it out on a broad scale.

Bitmovin needed a database with certain characteristics:

  • A database suitable for huge volumes of data storage and real-time analytics
  • A database that could be operated cost-effectively
  • Easy to scale
  • Be able to process structured and semi-structured data (such as log files) in high quantities and at very high throughput speeds
  • SQL as query language

The databases used up to this point would have been beyond the cost of this project and would not have made sense from a business perspective, so Bitmovin began looking for an alternative database solution.

 

Solution

Bitmovin uses CrateDB as the backend database for its video analytics component. Unlike other CrateDB customers, the company has not opted for a Database-as-a-Service solution in the cloud. The reason is simple: Bitmovin has always managed its very complicated technology stack itself, and also handled the implementation of CrateDB. The CrateDB team provides ongoing support and the technology itself is easy to use from installation to querying. There is no need for a complex architecture or advanced knowledge of SQL to get started.
"Getting Crate's support was a key enabler for us, having their team's support is very helpful."
Daniel Hölbling-Inzko Senior Director of Engineering – Analytics Bitmovin
cr-quote-img-white

The size of Bitmovin's backend is constantly increasing, with the number of nodes doubling since implementation. This is mainly because the system is exposed to a growing data load.

Currently, the database solution includes 14 nodes and 140 terabytes of storage, where the company stores structured data such as user events and user interactions. Every day, the video analytics tool adds around one billion new lines of code, with the largest tables comprising around 60 billion playback events. It is foreseeable that the database will continue to grow in the future.

 

Nodes Terabytes of storage Events in the largest tables New events per day
14 140,000 ~60 billion ~2 billion

 

Bitmovin works closely with the CrateDB team in times of great load peaks, to jointly integrate the insights gained from the increased data load into the product in a timely manner. With the support of CrateDB’s experts, Bitmovin is able to optimally adjust the configuration of the cluster ahead of events with high data load requirements.

 

Benefits

The most important and essential benefit of using CrateDB is that without it, Bitmovin would not be able to offer the video analytics tool at all. Alternative solutions are either not powerful enough or too expensive, so the analytics offering would not have been profitable. Thanks to CrateDB, the company has been able to increase its number of customers, as well as, its revenue. Furthermore, the database is stable and easy to manage, which frees up Bitmovin's developers to focus on more valuable tasks than building a scalable database system.

For Bitmovin, CrateDB is better than comparable products because:

  • The data aggregation performance required by the video analytics component would not be possible with other comparable solutions
  • The capability of scaling the clusters of the database itself
  • Bitmovin would like to accelerate this adaptation to peak loads even further by implementing logical replication, a new feature of CrateDB
  • With CrateDB, real-time video analysis and filling the dashboards with data is faster
"It is through the use of CrateDB that we are able to offer our large-scale video analytics component in the first place. Comparable products are either not capable of handling the large flood of data or they are simply too expensive."
Daniel Hölbling-Inzko Senior Director of Engineering – Analytics Bitmovin
cr-quote-img-white
A perfect example in practice is that Bitmovin would install a second cluster for a large event, such as a global sport event like the World Cup, to handle the expected flood of data. Via logical replication, the collected data can then be transferred to the main cluster for storage and archiving. After the event, the company can then shut down the second cluster again.
"Thanks to our collaboration with the CrateDB team on events that cause peak loads, our customers can count on the reliability of our services, even with very large audiences."
Daniel Hölbling-Inzko Senior Director of Engineering – Analytics Bitmovin
cr-quote-img-white
DISCUSSION - RECORDING
How Bitmovin uses CrateDB to monitor the biggest live video events

In this talk, Daniel Hölbling-Inzko explains how Bitmovin uses CrateDB to monitor the most significant live video events and especially which features they are using to address their monitoring and scalability challenges.

May 14, 2023

Interested in learning more?

Talk to an expert