Session

Session object

class graphscope.Session(**kwargs)[source]

A class for interacting with GraphScope graph computation service cluster.

A Session object encapsulates the environment in which Operation objects are executed/evaluated.

A session may own resources. It is important to release these resources when they are no longer required. To do this, invoke the close() method on the session.

A Session can register itself as default session with as_default(), and all operations after that will use the default session. Session deregister itself as a default session when closed.

The following example demonstrates its usage:

>>> import graphscope as gs

>>> # use session object explicitly
>>> sess = gs.session()
>>> g = sess.g()
>>> pg = g.project(vertices={'v': []}, edges={'e': ['dist']})
>>> r = gs.sssp(g, 4)
>>> sess.close()

>>> # or use a session as default
>>> sess = gs.session().as_default()
>>> g = gs.g()
>>> pg = g.project(vertices={'v': []}, edges={'e': ['dist']})
>>> r = gs.sssp(pg, 4)
>>> sess.close()

We support setup a service cluster and create a RPC session in following ways:

  • GraphScope graph computation service run in cluster managed by kubernetes.

    >>> s = graphscope.session()
    

    Also, Session provides several keyword params for users to define the cluster. You may use the param k8s_gs_image to specify the image for all engine pod, and param k8s_engine_cpu or k8s_engine_mem to specify the resources. More, you can find all params detail in __init__() method.

    >>> s = graphscope.session(
    ...         k8s_gs_image="registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:latest",
    ...         k8s_vineyard_cpu=0.1,
    ...         k8s_vineyard_mem="256Mi",
    ...         vineyard_shared_mem="4Gi",
    ...         k8s_engine_cpu=0.1,
    ...         k8s_engine_mem="256Mi")
    
  • or all params can be provided by a json configuration file or configuration dict.

    >>> s = graphscope.session(config='/tmp/config.json')
    >>> # Or
    >>> s = graphscope.session(config={'k8s_engine_cpu': 5, 'k8s_engine_mem': '5Gi'})
    
__enter__()[source]

Context management protocol. Returns self and register self as default session.

__exit__(type, value, traceback)[source]

Deregister self from the default session, close the session and release the resources, ignore all exceptions in close().

__init__(config=None, addr=None, mode='eager', cluster_type='k8s', num_workers=2, preemptive=True, k8s_namespace=None, k8s_service_type='NodePort', k8s_gs_image='registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.5.0', k8s_etcd_image='quay.io/coreos/etcd:v3.4.13', k8s_gie_graph_manager_image='registry.cn-hongkong.aliyuncs.com/graphscope/maxgraph_standalone_manager:0.5.0', k8s_zookeeper_image='zookeeper:3.4.14', k8s_image_pull_policy='IfNotPresent', k8s_image_pull_secrets=[], k8s_coordinator_cpu=0.5, k8s_coordinator_mem='512Mi', k8s_etcd_num_pods=1, k8s_etcd_cpu=1.0, k8s_etcd_mem='512Mi', k8s_zookeeper_cpu=0.2, k8s_zookeeper_mem='512Mi', k8s_gie_graph_manager_cpu=0.2, k8s_gie_graph_manager_mem='512Mi', k8s_vineyard_daemonset='none', k8s_vineyard_cpu=0.2, k8s_vineyard_mem='512Mi', vineyard_shared_mem='4Gi', k8s_engine_cpu=0.2, k8s_engine_mem='1Gi', k8s_mars_worker_cpu=0.2, k8s_mars_worker_mem='512Mi', k8s_mars_scheduler_cpu=0.2, k8s_mars_scheduler_mem='512Mi', k8s_volumes={}, k8s_waiting_for_delete=False, timeout_seconds=600, dangling_timeout_seconds=600, with_mars=False, enable_gaia=False, **kw)[source]

Construct a new GraphScope session.

Parameters
  • config (dict or str, optional) – The configuration dict or file about how to launch the GraphScope instance. For str, it will identify it as a path and read the configuration file to build a session if file exist. If not specified, the global default configuration DEFAULT_CONFIG_FILE will be used, which get value of GS_CONFIG_PATH in environment. Note that it will overwrite explicit parameters. Defaults to None.

  • addr (str, optional) – The endpoint of a pre-launched GraphScope instance with ‘<ip>:<port>’ format. A new session id will be generated for each session connection.

  • mode (str, optional) –

    optional values are eager and lazy. Defaults to eager. Eager execution is a flexible platform for research and experimentation, it provides:

    An intuitive interface: Quickly test on small data. Easier debugging: Call ops directly to inspect running models and test changes.

    Lazy execution means GraphScope does not process the data till it has to. It just gathers all the

    information to a DAG that we feed into it, and processes only when we execute sess.run(fetches)

  • cluster_type (str, optional) – Deploy GraphScope instance on hosts or k8s cluster. Defaults to k8s. Available options: “k8s” and “hosts”. Note that only support deployed on localhost with hosts mode.

  • num_workers (int, optional) – The number of workers to launch GraphScope engine. Defaults to 2.

  • preemptive (bool, optional) – If True, GraphScope instance will treat resource params (e.g. k8s_coordinator_cpu) as limits and provide the minimum available value as requests, but this will make pod has a Burstable QOS, which can be preempted by other pods with high QOS. Otherwise, it will set both requests and limits with the same value.

  • k8s_namespace (str, optional) – Contains the namespace to create all resource inside. If param missing, it will try to read namespace from kubernetes context, or a random namespace will be created and deleted if namespace not exist. Defaults to None.

  • k8s_service_type (str, optional) – Type determines how the GraphScope service is exposed. Valid options are NodePort, and LoadBalancer. Defaults to NodePort.

  • k8s_gs_image (str, optional) – The GraphScope engine’s image.

  • k8s_etcd_image (str, optional) – The image of etcd, which used by vineyard.

  • k8s_image_pull_policy (str, optional) – Kubernetes image pull policy. Defaults to “IfNotPresent”.

  • k8s_image_pull_secrets (list[str], optional) – A list of secret name used to authorize pull image.

  • k8s_gie_graph_manager_image (str, optional) – The GraphScope interactive engine’s graph manager image.

  • k8s_zookeeper_image (str, optional) – The image of zookeeper, which used by GIE graph manager.

  • k8s_vineyard_daemonset (str, optional) – The name of vineyard Helm deployment to use. GraphScope will try to discovery the daemonset from kubernetes cluster, then use it if exists, and fallback to launching a bundled vineyard container otherwise.

  • k8s_vineyard_cpu (float, optional) – Minimum number of CPU cores request for vineyard container. Defaults to 0.5.

  • k8s_vineyard_mem (str, optional) – Minimum number of memory request for vineyard container. Defaults to ‘512Mi’.

  • vineyard_shared_mem (str, optional) – Init size of vineyard shared memory. Defaults to ‘4Gi’.

  • k8s_engine_cpu (float, optional) – Minimum number of CPU cores request for engine container. Defaults to 0.5.

  • k8s_engine_mem (str, optional) – Minimum number of memory request for engine container. Defaults to ‘4Gi’.

  • k8s_coordinator_cpu (float, optional) – Minimum number of CPU cores request for coordinator pod. Defaults to 1.0.

  • k8s_coordinator_mem (str, optional) – Minimum number of memory request for coordinator pod. Defaults to ‘4Gi’.

  • k8s_etcd_num_pods (int, optional) – The number of etcd pods. Defaults to 3.

  • k8s_etcd_cpu (float, optional) – Minimum number of CPU cores request for etcd pod. Defaults to 0.5.

  • k8s_etcd_mem (str, optional) – Minimum number of memory request for etcd pod. Defaults to ‘128Mi’.

  • k8s_zookeeper_cpu (float, optional) – Minimum number of CPU cores request for zookeeper container. Defaults to 0.5.

  • k8s_zookeeper_mem (str, optional) – Minimum number of memory request for zookeeper container. Defaults to ‘256Mi’.

  • k8s_gie_graph_manager_cpu (float, optional) – Minimum number of CPU cores request for graphmanager container. Defaults to 1.0.

  • k8s_gie_graph_manager_mem (str, optional) – Minimum number of memory request for graphmanager container. Defaults to ‘4Gi’.

  • k8s_mars_worker_cpu (float, optional) – Minimum number of CPU cores request for mars worker container. Defaults to 0.5.

  • k8s_mars_worker_mem (str, optional) – Minimum number of memory request for mars worker container. Defaults to ‘4Gi’.

  • k8s_mars_scheduler_cpu (float, optional) – Minimum number of CPU cores request for mars scheduler container. Defaults to 0.5.

  • k8s_mars_scheduler_mem (str, optional) – Minimum number of memory request for mars scheduler container. Defaults to ‘2Gi’.

  • with_mars (bool, optional) – Launch graphscope with mars. Defaults to False.

  • enable_gaia (bool, optional) – Launch graphscope with gaia enabled. Defaults to False.

  • k8s_volumes (dict, optional) –

    A dict of k8s volume which represents a directory containing data, accessible to the containers in a pod. Defaults to {}.

    For example, you can mount host path with:

    k8s_volumes = {
    “my-data”: {

    “type”: “hostPath”, “field”: {

    ”path”: “<path>”, “type”: “Directory”

    }, “mounts”: [

    {

    “mountPath”: “<path1>”

    }, {

    ”mountPath”: “<path2>”

    }

    ]

    }

    }

    Or you can mount PVC with:

    k8s_volumes = {
    “my-data”: {

    “type”: “persistentVolumeClaim”, “field”: {

    ”claimName”: “your-pvc-name”

    }, “mounts”: [

    {

    “mountPath”: “<path1>”

    }

    ]

    }

    }

    Also, you can mount a single volume with:

    k8s_volumes = {
    “my-data”: {

    “type”: “hostPath”, “field”: {xxx}, “mounts”: {

    ”mountPath”: “<path1>”

    }

    }

    }

  • timeout_seconds (int, optional) – For waiting service ready (or waiting for delete if k8s_waiting_for_delete is True).

  • dangling_timeout_seconds (int, optional) – After seconds of client disconnect, coordinator will kill this graphscope instance. Defaults to 600. Expect this value to be greater than 5 (heartbeat interval). Disable dangling check by setting -1.

  • k8s_waiting_for_delete (bool, optional) – Waiting for service delete or not. Defaults to False.

  • **kw (dict, optional) –

    Other optional parameters will be put to **kw. - k8s_minikube_vm_driver: Deprecated.

    • k8s_client_config (dict, optional):

      Provide configurable parameters for connecting to remote k8s, which strongly relies on the kube_config.new_client_from_config function. eg: {“config_file”: “~/.kube/config”, “context”: None, “persist_config”: True} config_file: Name of the kube-config file. context: set the active context. If is set to None, current_context from config file will be used. persist_config: If True, config file will be updated when changed(e.g GCP token refresh).

    • log_level: Deprecated.

      Move this param as a global configuration. Set via graphscope.set_option(log_level=’DEBUG’)

    • show_log: Deprecated.

      Move this param as a global configuration.Set via graphscope.set_option(show_log=True)

    • k8s_vineyard_shared_mem: Deprecated.

      Please use vineyard_shared_mem instead.

Raises

TypeError – If the given argument combination is invalid and cannot be used to create a GraphScope session.

as_default()[source]

Obtain a context manager that make this object as default session.

This method is used when a Session is constructed, which will immediately install self as a default session.

Raises

ValueError – If default session exist in current context.

Returns

A context manager using this session as the default session.

close()[source]

Closes this session.

This method frees all resources associated with the session.

property engine_config

Show the engine configration associated with session in json format.

get_config()[source]

Get configuration of the session.

gremlin(graph, engine_params=None)[source]

Get a interactive engine handler to execute gremlin queries.

It will return a instance of graphscope.interactive.query.InteractiveQueryDAGNode, that will be evaluated by :method:`sess.run` in eager mode.

Note that this method will be executed implicitly in eager mode when a property graph created and cache a instance of InteractiveQuery in session if initializing_interactive_engine is True. If you want to create a new instance under the same graph by different params, you should close the instance first.

>>> # close and recreate InteractiveQuery in eager mode.
>>> interactive_query = sess.gremlin(g)
>>> interactive_query.close()
>>> interactive_query = sess.gremlin(g, engine_params={"xxx":"xxx"})
Parameters
  • graph (graphscope.framework.graph.GraphDAGNode) – The graph to create interactive instance.

  • engine_params (dict, optional) – Configure startup parameters of interactive engine. You can also configure this param by graphscope.set_option(engine_params={}). See a list of configurable keys in interactive_engine/deploy/docker/dockerfile/executor.vineyard.properties

Raises

InvalidArgumentError

  • graph is not a property graph. - graph is unloaded in eager mode.

Returns

InteractiveQuery to execute gremlin queries, evaluated in eager mode.

Return type

graphscope.interactive.query.InteractiveQueryDAGNode

property info

Show all resources info associated with session in json format.

learning(graph, nodes=None, edges=None, gen_labels=None)[source]

Start a graph learning engine.

Parameters
  • nodes (list) – The node types that will be used for gnn training.

  • edges (list) – The edge types that will be used for gnn training.

  • gen_labels (list) – Extra node and edge labels on original graph for gnn training.

Returns

An instance of learning graph that could be feed to the learning engine, evaluated in eager node.

Return type

graphscope.learning.GraphDAGNode

load_from(*args, **kwargs)[source]

Load a graph within the session. See more information in graphscope.load_from().

Session Functions

graphscope.session

alias of graphscope.client.session.Session

graphscope.get_default_session()

Returns the default session for the current context.

graphscope.set_option(**kwargs)

Set the value of specified options.

graphscope.get_option(key)

Get the value of specified option.