ShardingSphere Operator Practical Guide · ShardingSphere - Blog

ShardingSphere Operator Practical Guide

Apache ShardingSphere-on-Cloud sub-project

ShardingSphere-on-Cloud is a sub-project of Apache ShardingSphere for cloud solutions, covering verything from automated deployment scripts to virtual machines in AWS, GCP, AliCloud, Huawei Cloud cloud environments such as CloudFormation Stack templates, Terraform and more.

The project also covers tools such as Helm Charts, Operator, and automatic horizontal scaling in Kubernetes environments, and will gradually cover ShardingSphere high-availability, observability, and security compliance in cloud environments.

For more information, please refer to the official documentation.

Why Do You Need ShardingSphere Operator

Kubernetes provides a way to implement a “platform on top of a platform”, the Operator pattern, which leverages a custom CustomResourceDefinition with the Kubernetes exposed Reconcile model framework.

The Reconcile model framework exposed by Kubernetes enables developers to quickly implement a declarative custom Ops tool for a given application, such as Prometheus-Operator, Nats-Operator, etc.

As a transparent database enhancement engine, Apache ShardingSphere supports access to any client using MySQL, PostgreSQL, openGauss protocols, with ShardingSphere-Proxy being more suitable to heterogeneous languages and high-availability scenarios.

Using Operator makes the deployment and maintenance of ShardingSphere-Proxy on Kubernetes much easier and more efficient.


Using ShardingSphere Operator: An Example

Before using ShardingSphere, you will need to prepare one or more compatible databases as storage nodes.

Detailed instructions can be found in the official documentation for the ShardingSphere-on-Cloud sub-project.


Install ShardingSphere-Operator

Find the required configuration content and configuration file directory for installation here.

To facilitate the installation of Operator, our community provides an online installation in addition to the source code installation. The two methods are as follows:

Online Installation

kubectl create ns shardingsphere-operator
	helm repo add shardingsphere
	helm repo update
	helm install demo-release shardingsphere/apache-shardingsphere-operator-charts -n shardingsphere-operator

Example of operation result:


Note: please refer to the official documentation for source code installationใ€‚

At this point, you can see that the operator has injected the crd that Operator will be working on into the Kubernetes cluster by using kubectl get crd:


As well as viewing the deployed ShardingSphere-Operator:


Deploy ShardingSphere-Proxy Clusters

For a list of all the parameters that can be configured options, refer to the documentation here.

Tip: you need to provide an accessible ZooKeeper cluster before you can run ShardingSphere-Proxy.

To run a ShardingSphere-Proxy you need to write two CustomResourceDefinition files: shardingsphereproxy.yaml and shardingsphereproxyserverconfigs.yaml, examples of which are as follows:

# shardingsphereproxy.yaml
kind: ShardingSphereProxy
    app: shardingsphere-proxy
  name: shardingsphere-proxy
  namespace: shardingsphere-demo
    version: 5.1.47
  port: 3307
  proxyConfigName: shardingsphere-proxy-configuration
  replicas: 3
    type: ClusterIP
  version: 5.3.0
# shardingsphereproxyserverconfigs.yaml
kind: ShardingSphereProxyServerConfig
    app: shardingsphere-proxy
  name: shardingsphere-proxy-configuration
  namespace: shardingsphere-demo
      type: ALL_PERMITTED
    - password: root
      user: root@%
        maxRetries: 3
        namespace: governance_ds
        operationTimeoutMilliseconds: 5000
        retryIntervalMilliseconds: 500
        server-lists: shardingsphere-proxy-zookeeper.shardingsphere:2181 # This is an example and it should be modified to the real scenario.
        timeToLiveSeconds: 600
      type: ZooKeeper
    type: Cluster
    proxy-frontend-database-protocol-type: MySQL

Operation examples:




Then you can check the status of the Pod running under the shardingsphere-demo namespace:


Related resources created by the Operator can also be found at:


Configure ShardingSphere-Proxy for Data Encryption Capabilities

ShardingSphere supports DistSQL to help DBAs quickly set up and run storage node registration, rule configuration and more. You can find out more here.

As enterprises continue to transform digitally, increasing amounts of user data are being communicated between all kinds of businesses. Various countries and regions have introduced regulations and frameworks to ensure data security, such as the EU’s GDPR and others.

Apache ShardingSphere offers multiple functions in data security such as data encryption and decryption.

Followings are examples of data encryption:

  1. Register storage nodes


  1. Create encryption rules


  1. Create logical tables


  1. View physical table properties


  1. Logical table insertion and query


  1. Data source physical table query

You can find further details of the encryption features here.

Good news! Our new ShardingSphere-on-Cloud is now live!

With the migration of the ShardingSphere-on-Cloud sub-project to ShardingSphere-on-Cloud Github, we designed and launched a new website.

As the project grows, this will facilitate interested users or contributors better understand the project and participate in the community, including documentation for each version, an introduction to the community, information about the Apache Foundation, community updates, information about the Apache Foundation, and etc.


Relevant Links:

๐Ÿ”— ShardingSphere-on-Cloud Github

๐Ÿ”— ShardingSphere-on-Cloud Official Website

๐Ÿ”— Apache ShardingSphere GitHub

๐Ÿ”— Apache ShardingSphere Official Website

๐Ÿ”— Apache ShardingSphere Slack Channel