How to Deploy GraphScope by Helm on Kubernetes

on Tech

Deploy-GraphScope-by-helm This article describes how to deploy and use GraphScope clusters using the Helm tool. Helm is a software package management tool in the K8s ecosystem, similar to Ubuntu’s apt or Python’s pip, designed for managing K8s application resources. Using Helm, you can easily package, distribute, install, upgrade, and rollback kubernetes applications. GraphScope also supports deployment by Helm.

Prerequisites

Before you start, make sure that the current environment has an K8s cluster. If not, you can refer to previous articles for installation. In addition, you also need to install the Python package graphscope_client to run the Python examples in this article,

$ pip3 install graphscope_client

Install Helm

According to the official Helm documentation, Helm can be installed using the following command.

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

If the server cannot connect to the network, you can pre-download Helm in an environment with a network connection, and then copy it to the server.

Here is an example of installing Helm v3.8.1:

$ wget https://get.helm.sh/helm-v3.8.1-linux-amd64.tar.gz
tar zxvf helm-v3.8.1-linux-amd64.tar.gz

After decompression, you can copy the helm file to the server by yourself and put it under the /usr/local/bin directory. Then execute

$ helm version
version.BuildInfo{Version:"v3.5.4", GitCommit:"1b5edb69df3d3a08df77c9902dc17af864ff05d1", GitTreeState:"clean", GoVersion:"go1.15.11"}

If commands succeed, it means that Helm is installed successfully.

Configure Helm repo

GraphScope’s Helm Charts are hosted in the repository https://graphscope.oss-cn-beijing.aliyuncs.com/, so you can add this repository using the following command:

$ helm repo add graphscope https://graphscope.oss-cn-beijing.aliyuncs.com/charts/
"graphscope" has been added to your repositories

After adding it, you can use helm search to search for Charts provided by GraphScope.

$ helm search repo graphscope
NAME                       	CHART VERSION	APP VERSION	DESCRIPTION
graphscope/graphscope      	0.22.0       	0.22.0     	A One-Stop Large-Scale Graph Computing System f...
graphscope/graphscope-store	0.22.0       	3.4.0      	GraphScope Store is a disk-based row-oriented m...
graphscope/gie-standalone  	0.20.0       	3.4.0      	Chart to Deploy GIE on Vineyard Storage

Common operations of Helm

The common operations of Helm include installing (helm install), upgrading (helm upgrade), uninstalling (helm uninstall), and listing (helm ls). Often, a Helm Chart has many parameters that need to be configured. Configuring them through command line is cumbersome, so it is recommended to use YAML files to configure these parameters. In this article, a file named values.yaml is used to configure charts, which is also a naming convention in the Helm community.

Install GraphScope Chart

To install charts, you must specify the name of the Chart (chart_name) and the name of the deployed instance (release_name). You can also specify one or more values.yaml files to configure Chart. If you have specific requirements for the version of Chart, you can specify the corresponding version (chart_version, default is the latest version) through the --version parameter. The basic command is as follows:

$ helm install demo graphscope/graphscope [--namespace=default] [--version=0.22.0]

The above command will installed version 0.22.0 of GraphScope instance under default namespace, and the instance name is demo.

Get the address of GraphScope service

Because images will be pulled during the first installation of Chart, you need to wait a few minutes. During this time, you can observe the status by executing helm test command multiple times:

$ helm test demo
NAME: demo
LAST DEPLOYED: Sat Jun 10 17:08:49 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE:     test-rpc-service-demo
Last Started:   Sat Jun 10 17:14:21 2023
Last Completed: Sat Jun 10 17:14:25 2023
Phase:          Succeeded
NOTES:
The GraphScope has been deployed.

If the status output is as shown above, it means that the local GraphScope instance has been successfully installed. Next, take default namespace as an example. At this point, you can get the service address of the current instance:

$ export NODE_IP=$(kubectl --namespace default  get pod -l graphscope.coordinator.name=coordinator-demo -ojsonpath="{.items[0].status.hostIP}")
$ export NODE_PORT=$(kubectl --namespace default get services coordinator-service-demo  -ojsonpath="{.spec.ports[0].nodePort}")
$ echo "GraphScope service listen on ${NODE_IP}:${NODE_PORT}"

GraphScope service listen on 192.168.0.65:30262

The above command is cumbersome, but don’t worry. After GraphScope Chart is successfully installed, these commands will be displayed on the console for copy and paste. You could also use helm status command at any time to get these output incase you lost it.

Connect to service using client

After obtaining the service address (192.168.0.65:30262 in the above command), you can use GraphScope Python client to connect to this service:

>>> import graphscope
>>> graphscope.set_option(show_log=True)
>>> # sess = graphscope.session(addr='<ip>:<port>')
>>> sess = graphscope.session(addr='192.168.0.65:30262')
Connecting graphscope session with address: 192.168.0.65:30262
GraphScope coordinator service connected.

Unlike the way of deploying GraphScope through Python client, the lifecycle of GraphScope instances deployed using Helm tools are not bound to the client’s lifecycle. Therefore, you can connect to a GraphScope instance deployed using Helm multiple times through the client, but only one client can connect to the service at a time.

>>> import graphscope
>>> sess1 = graphscope.session(addr='192.168.0.65:30262')
>>> sess1.close()  # The second session cannot connect to server until previous one was closed.
>>> sess2 = graphscope.session(addr='192.168.0.65:30262')

Uninstall GraphScope instance

Uninstall an instance only requires the corresponding instance name:

$helm uninstall demo

Parameter configuration

Deploying GraphScope through Helm can also configure parameters for customizing the GraphScope instance, such as CPU, memory, and volumes. The parameter configurations currently supported can be found in values.yaml file. Here let’s take mounting datasets as an example to illustrate how to configure GraphScope Chart through values.yaml, which is one of the most common situations in use. First, create a values.yaml file with the following content:

volumes:
  enabled: true
  items:
    data:
      type: hostPath
      field:
        type: Directory
        path: /data
      mounts:
      - mountPath: /tmp/data

The above description volumes will use K8s HostPath type volume and mount the /data path of the corresponding node to the /tmp/data path of GraphScope instance. For more details about mounting volumes, please refer to K8s official documentation.

Next, we can specify the values.yaml as configuration file when installing GraphScope Chart:

$ helm install demo graphscope/graphscope -f values.yaml

Other resource configurations, such as CPU and memory, can also be configured by referring to GraphScope configuration file.

Offline use of Helm charts

If the server does not have external network access, it is impossible to install GraphScope by configuring Helm repo. In this case, you need to download chart files required for cluster installation on a machine with external network access and then copy them to the server. Taking version 0.22.0 as an example, you could download the chart as follows:

$ wget https://graphscope.oss-cn-beijing.aliyuncs.com/charts/graphscope-0.22.0.tgz

After downloading is complete, you can copy it to the corresponding directory on the server and decompress it, then you can use these charts to install GraphScope instances through helm install command. The command is as follows.

$ tar zxvf graphscope-0.22.0.tgz
$ helm install demo ./graphscope

Note that during installation of instances, images need to be pre-downloaded on servers. You could use docker save to save the contents of the image to a tarball, and copy that tarball to the servers then use docker load to extract the content as image.

Conclusion

This article introduces how to connect GraphScope client to a GraphScope cluster deployed using Helm tools. In addition, GraphScope will also allow users to deploy, operate and use GraphScope on Kubernetes more easily through GraphScope Operator in the future. Stay tuned for more updates from GraphScope.