Published on: 2026-07-15
Source: Novosibirsk State University –
An important disclaimer is at the bottom of this article.
Graduate Faculty of Information Technology of Novosibirsk State University Mark Buchner improved the performance of open-source software used by large companies worldwide for recording into distributed databases of terabyte scale. His changes are already included in the official versions of the program and allow accelerating data recording depending on the operating mode by approximately 2–4 times. This concerns systems that in real time collect readings from thousands of sensors from millions of devices – from cars to “smart” household appliances.
According to Buchner, his master’s thesis was dedicated to “the research and implementation of cross-platform data representation formats to accelerate a distributed database.” In fact, he was engaged in modernizing existing software. It concerns Apache Hudi platform, which belongs to the family of lakehouse systems (lakehouse; allows storing, managing, and analyzing data).
— The essence of the work is to record data faster in a very large database that stores not just gigabytes of data, but terabytes of data, — he said.
The program is used specifically by companies, not individual users: they need solutions that can handle a constant stream of telemetry and events from a large number of devices.
For example, modern cars, especially electric vehicles and hybrids, are equipped with numerous sensors and constantly send data on system status and operating parameters to the manufacturer’s servers. If there are millions of such cars, each with thousands of sensors, companies have to “land” colossal volumes of information into databases every second in order to then look for deviations, patterns, potential faults, and optimize service.
— I spent almost two years working on accelerating them and recording this data faster, — Mark explained.
According to his estimate, depending on the configuration and mode of operation, the acceleration ranged from two to four times.
Similar challenges exist not only in automotive manufacturing but also in industry, logistics, energy, and in the segment of “smart” household appliances. More and more devices are connecting to the network, and built-in sensors continuously send telemetry to the manufacturer or service company. The faster and more reliably the database receives and processes this stream, the more promptly the business can respond to problems, plan maintenance, improve products, and enhance safety. For large corporations, this is already a matter not of convenience but of competitiveness: delays in recording and analyzing data limit the capabilities of predictive analytics and the development of services based on artificial intelligence.
The feature of the software improved by Buchner is that it is distributed with open source code and does not belong to any one corporation. Any company can implement it into its infrastructure. The changes proposed by the NSU graduate were reviewed by the developers and officially accepted.
— The committee said, “Yes, this is a good change, a good improvement. Performance is really improving.” — he noted.
The improved code has now been incorporated into certain versions of the software, and the companies that update it automatically gain increased speed in data processing.
According to Buchner, this software is currently actively used by foreign companies in the Asian market. The developer does not have data on whether Russian firms use it:
— As I understand it, no, they do not have such a task yet. But in the future, it will probably definitely arise.
Mark himself considers his part of the work completed: first he worked on speeding up data reading, then on speeding up writing.
— I have sped up everything that is already possible in these programs, it seems to me, and now it’s time to focus on something else, — he concluded.
Please note; This information is raw content obtained directly from the information source. It is an accurate report of what the source claims and does not necessarily reflect the position of MIL-OSI or its clients.