Skip to content

Article Selections

Managing Metadata in Sharded Database Environments with ShardingSphere's Built-in Metadata Function

Apache ShardingSphere is a popular open-source data management platform that supports sharding, encryption, read/write splitting, transactions, and high availability. The metadata of ShardingSphere comprises of rules, data sources, and table structures, which are crucial for the platform’s proper functioning. ShardingSphere uses the governance center’s capabilities, such as ZooKeeper and etcd, for the sharing and modification of cluster configurations to achieve the horizontal expansion of computing nodes. In this blog post, we will focus on understanding the metadata structure of Apache ShardingSphere. We will explore the three-layer metadata structure of ShardingSphere in ZooKeeper, which includes the metadata information, built-in metadata database, and simulated MySQL database.

Read more →

Streamlining Database Configuration with DistSQL's Export, Import, and YAML Conversion

DistSQL, which stands for Distributed SQL, is a specialized operating language exclusive to Apache ShardingSphere. This language provides users with a simplified and powerful dynamic management system that allows them to operate ShardingSphere like a traditional database. One of the key benefits of using DistSQL is the ability to define resources and rules online without the need to modify YAML files and restart the system. This streamlines the management process and reduces the potential for errors.

Read more →

ShardingSphere Operator Practical Guide

Apache ShardingSphere-on-Cloud sub-project ShardingSphere-on-Cloud is a sub-project of Apache ShardingSphere for cloud solutions, covering verything from automated deployment scripts to virtual machines in AWS, GCP, AliCloud, Huawei Cloud cloud environments such as CloudFormation Stack templates, Terraform and more. The project also covers tools such as Helm Charts, Operator, and automatic horizontal scaling in Kubernetes environments, and will gradually cover ShardingSphere high-availability, observability, and security compliance in cloud environments. If you want to quickly understand, validate or use the features of ShardingSphere-Proxy and don’t have a Kubernetes environment, you can use AWS CloudFormation or Terraform for on-demand deployments, as described in the previous article “Use AWS CloudFormation to create ShardingSphere HA clusters”. If you’d like to deploy in a Kubernetes environment, you can leverage the Operator feature or install ShardingSphere-Proxy directly without using Operator but with Helm Charts. For more information, please refer to the official documentation.

Read more →

ShardingSphere 5.3.x Update: Spring Configuration Update Guide

Before 5.3.0, ShardingSphere-JDBC supported Java API, YAML, Spring Boot Starter, and Spring Namespace. However, compatibility with Spring has brought the following challenges to the community: Many configuration files need to be adjusted when adding or updating API, which causes a heavy workload. The community needs to maintain lots of configuration documents and examples. Spring Bean life cycle management is susceptible to other project dependencies, such as PostProcessor.[1][2] Spring Boot Starter and Spring Namespace’s style differs from YAML in ShardingSphere. Spring Boot Starter and Spring Namespace are affected by the update of Spring, causing more configuration compatibility problems. For example, in the latest release of Spring Boot 3.0.0,

Read more →

ShardingSphere-Proxy Database Protocol Interaction Explained

Database protocols are considered relatively unknown knowledge for most developers. Generally, users and developers use databases through off-the-shelf database clients and drivers, without manipulating database protocols directly. However, having some basic understanding of the features and processes of database protocols can help developers provide some ideas to identify troubleshooting problems when it comes to database functionality and performance issues. In this post, we will introduce the features of commonly used open source database protocols such as MySQL and PostgreSQL, and explain how ShardingSphere-Proxy interacts with clients at the database protocol level.

Read more →

Dynamic Data Masking | ShardingSphere 5.3.1 Deep Dive

With increased focus placed on user data protection and the introduction of privacy laws or data protection regulations such as GDPR, being compliant in terms of digital personal privacy is now of paramount importance. Traditional applications generally lack personal data protection. Data masking enables data returned by production databases to be specially masked, encrypted, hidden, and audited according to the users’ role, responsibility, and other defined rules without having to make any changes to the data.

Read more →

A Distributed Database Load Balancing Architecture Based on ShardingSphere: Demo and User Case

This post introduces how to build a distributed database load-balancing architecture based on ShardingSphere, and the impact of introducing load balancing with a user case. Finally, a one-stop solution for Apache ShardingSphere distributed databases on the cloud will be presented and demonstrated at the end. ShardingSphere-based distributed database load balancing architecture ShardingSphere Load Balancing Architecture Essentials As many of our readers may already know, Apache ShardingSphere is a distributed database ecosystem that transforms any database into a distributed database and enhances it with data sharding, elastic scaling, encryption, and other capabilities.

Read more →

Leverage Terraform to Create an Apache ShardingSphere Proxy High Availability Cluster on AWS

For this ShardingSphere blog entry, we want you to take advantage of the benefits of IaC (Infrastructure as Code) by deploying and managing a ShardingSphere Proxy cluster the IaC way. With this in mind, we plan to use Terraform to create a ShardingSphere-Proxy high availability cluster with multiple availability zones. Background Terraform Terraform is Hashicorp’s open source infrastructure automation orchestration tool that uses IaC philosophy to manage infrastructure changes.

Read more →

Refactoring the DistSQL Syntax | ShardingSphere 5.3.0 Deep Dive

Background DistSQL (Distributed SQL) is ShardingSphere’s SQL-like functional language. Since we released version 5.0.0-Beta, we’ve been iterating rapidly and providing users with features including rule management, cluster management, and metadata management. It was an incremental improvement process including many steps. At the same time, DistSQL is still releatively young. The ShardingSphere community often receives fresh ideas and suggestions about DistSQL, which means fast growth with lots of possible different development directions.

Read more →

Fuzzy query for CipherColumn | ShardingSphere 5.3.0 Deep Dive

1. Background Apache ShardingSphere supports data encryption. By parsing users’ SQL input and rewriting the SQL according to the users’ encryption rules, the original data is encrypted and stored with ciphertext data in the underlying database at the same time. When a user queries the data, it only fetches the ciphertext data from the database, decrypts it, and finally returns the decrypted original data to the user. However, because the encryption algorithm encrypts the whole string, fuzzy queries cannot be achieved.

Read more →