Back to Blog

2022 Distributed Database Learning Summary

2022-12-31

Introduction

2022 was the year I got into distributed database technology. With the explosive growth of data volume and rising business complexity, traditional monolithic databases were no longer able to meet the needs of modern applications, and distributed databases emerged as one of the solutions.

What I Learned

1. Distributed Database Fundamentals

In 2022, I started learning the foundational concepts of distributed databases:

  • Basic principles of distributed systems
  • The CAP theorem and BASE theory
  • Distributed transaction processing
  • Data sharding and replication
  • Consensus protocols (such as Raft and Paxos)

2. Mainstream Products

A deep dive into mainstream distributed database products:

  • TiDB: an open-source distributed HTAP database
  • PolarDB: Alibaba Cloud’s cloud-native database
  • OceanBase: Ant Group’s distributed relational database
  • CockroachDB: a cloud-native distributed SQL database
  • Cassandra: a NoSQL wide-column store database

3. Comparing Technical Characteristics

Studying the technical characteristics of different distributed databases:

  • HTAP (hybrid transactional/analytical processing) capability
  • Horizontal scalability
  • High availability and disaster recovery
  • Compatibility (such as MySQL compatibility)
  • Performance characteristics and suitable use cases

Hands-On Exploration

Alongside my studies, I did some hands-on exploration:

  • Learning TiDB’s architecture and deployment
  • Learning TiDB’s SQL optimization methods
  • Exploring monitoring and operations for distributed databases
  • Learning performance tuning for distributed databases
  • Learning about database certifications

Key Takeaways

1. Better Technical Understanding

  • Understood the design philosophy behind distributed databases
  • Recognized the challenges distributed systems face
  • Learned the factors to weigh when selecting a database for different scenarios

2. Broader Architectural Thinking

  • Learned to think about data storage from a system-architecture perspective
  • Recognized the trade-off between data consistency and availability
  • Improved my understanding of large-scale system design

Looking Ahead

My plan for 2023 is to keep going deeper into distributed database technology:

  • Hands-on practice with concrete products such as TiDB
  • Learning more about distributed database operations
  • Exploring how NewSQL and traditional databases can be combined
  • Following trends in cloud-native databases
  • Learning troubleshooting and performance optimization for distributed databases

As a key part of modern application architecture, distributed databases are well worth studying. I will keep following and learning about them, preparing myself for possible future system design work.