Graph Planner Interface by JNI and RESTful API¶

GraphPlanner is the primary entry point for GOpt’s query optimization and physical plan generation. Originally, it was tightly integrated into the Frontend service, where it optimized Cypher queries received via the Bolt protocol and generated execution plans for various backend engines.

To enhance its flexibility and ease of integration, GraphPlanner is now available as a standalone module, free from any dependencies on other Frontend modules. It supports both JNI and RESTful API interfaces, enabling lightweight and straightforward integration into diverse systems. Whether you are working on a native application or web-based services, GraphPlanner can seamlessly integrate into your architecture, providing efficient query optimization and physical plan generation across a wide range of use cases.

JNI API¶

Interface Overview¶

We provide a c++ wrapper implementation GraphPlannerWrapper for the JNI interface. Here is a brief explanation of the logical interface provided by the c++ class.

Constructor:

/**
 * @brief Constructs a new GraphPlannerWrapper object
 * @param java_path Java class path
 * @param jna_path JNA library path
 * @param graph_schema_yaml Path to the graph schema file in YAML format (optional)
 * @param graph_statistic_json Path to the graph statistics file in JSON format (optional)
 */
GraphPlannerWrapper(const std::string &java_path,
                    const std::string &jna_path,
                    const std::string &graph_schema_yaml = "",
                    const std::string &graph_statistic_json = "");

Method:

/**
 * @brief Compile a cypher query to a physical plan by JNI invocation.
 * @param compiler_config_path The path of compiler config file.
 * @param cypher_query_string The cypher query string.
 * @param graph_schema_yaml Content of the graph schema in YAML format
 * @param graph_statistic_json Content of the graph statistics in JSON format
 * @return The physical plan in bytes and result schema in yaml.
 */
Plan GraphPlannerWrapper::CompilePlan(const std::string &compiler_config_path,
                                      const std::string &cypher_query_string,
                                      const std::string &graph_schema_yaml,
                                      const std::string &graph_statistic_json)

Getting Started¶

Follow the steps below to get started with the Graph Planner interface for c++ invocation.

Step 1: Build the Project¶

Navigate to the project directory and build the package using Maven:

cd interactive_engine
mvn clean package -DskipTests -Pgraph-planner-jni

Step 2: Locate and Extract the Package¶

After the build completes, a tarball named graph-planner-jni.tar.gz will be available in the assembly/target directory. Extract the contents of the tarball:

cd assembly/target
tar xvzf graph-planner-jni.tar.gz
cd graph-planner-jni

Step 3: Run the Example Binary¶

To demonstrate the usage of the JNI interface, an example binary test_graph_planner is provided. Use the following command to execute it:

# bin/test_graph_planner <java class path> <jna lib path> <graph schema path> <graph statistics path> <query> <config path>
bin/test_graph_planner libs native ./conf/graph.yaml ./conf/modern_statistics.json "MATCH (n) RETURN n, COUNT(n);" ./conf/gs_interactive_hiactor.yaml

The output consists of three key fields:

Error code for plan compilation. The following table outlines the error codes, along with their corresponding descriptions:

Error Code	Description
OK	Query compilation succeeded.
GREMLIN_INVALID_SYNTAX	The provided Gremlin query contains syntax errors.
CYPHER_INVALID_SYNTAX	The provided Cypher query contains syntax errors.
TAG_NOT_FOUND	The specified tag is not found in the current query context.
LABEL_NOT_FOUND	The specified label is not found under the schema constraints.
PROPERTY_NOT_FOUND	The specified property is not found under the schema constraints.
TYPE_INFERENCE_FAILED	The query contains invalid graph patterns that violate schema constraints.
LOGICAL_PLAN_BUILD_FAILED	An error occurred during logical plan optimization. The error message provides specific details.
PHYSICAL_PLAN_BUILD_FAILED	An error occurred during physical plan construction.
GREMLIN_INVALID_RESULT	An error occurred while parsing the Gremlin query results.
CYPHER_INVALID_RESULT	An error occurred while parsing the Cypher query results.
ENGINE_UNAVAILABLE	The lower execution engine is unavailable.
QUERY_EXECUTION_TIMEOUT	The query execution time exceeded the predefined limit.
META_SCHEMA_NOT_READY	The schema metadata is not ready for querying.
META_STATISTICS_NOT_READY	The statistical metadata is not ready for querying.
EMPTY_RESULT	The compilation determines that the query will produce an empty result.

Physical plan in byte format, adheres to the specifications defined in the protobuf.
Result schema in YAML format. Below is an example of a result schema:
```
schema:
name: default
description: default desc
mode: READ
extension: .so
library: libdefault.so
params: []
returns:
- name: n
    type: {primitive_type: DT_UNKNOWN}
- name: $f1
    type: {primitive_type: DT_SIGNED_INT64}
type: UNKNOWN
query: MATCH (n) RETURN n, COUNT(n);
```
The returns field defines the structure of the data returned by backend engines. Each nested entry in the returns field includes three components:
- the column name, which specifies the name of the result column;
- the entry’s ordinal position, which determines the column ID;
- the type, which enforces the data type constraint for the column.

Restful API¶

We provide an alternative method to expose the interface as a RESTful API. Follow the steps below to access the interface via REST.

Getting Started¶

Step 1: Build the Project¶

To build the project, run the following command:

cd interactive_engine
# Use '-Dskip.native=true' to skip compiling C++ native code
mvn clean package -DskipTests -Pgraph-planner-jni -Dskip.native=true

Step 2: Locate and Extract the Package¶

Once the build completes, a tarball named graph-planner-jni.tar.gz will be available in the assembly/target directory. Extract the contents as follows:

cd assembly/target
tar xvzf graph-planner-jni.tar.gz
cd graph-planner-jni

Step 3: Start the Graph Planner RESTful Service¶

To start the service, run the following command:

java -cp ".:./libs/*" com.alibaba.graphscope.sdk.restful.GraphPlannerService --spring.config.location=./conf/application.yaml

Step 4: Access the RESTful API by `Curl`¶

To send a request to the RESTful API, use the following curl command:

curl -X POST http://localhost:8080/api/compilePlan \
    -H "Content-Type: application/json" \
    -d "{
        \"configPath\": \"$configPath\",
        \"query\": \"$query\",
        \"schemaYaml\": \"$schemaYaml\",
        \"statsJson\": \"$statsJson\"
    }"

Replace $configPath, $query, $schemaYaml, and $statsJson with the appropriate values.

The response will be in JSON format, similar to:

{
    "graphPlan": {
        "physicalBytes": "",
        "resultSchemaYaml": ""
    }
}

The response contains two fields:

physicalBytes: A Base64-encoded string representing the physical plan bytes.
resultSchemaYaml: A string representing the YAML schema.

You can decode these values into the required structures.

Step 4: Access the RESTful API by `Java` Sdk¶

Alternatively, if you are a java-side user, we provide a java sdk example to guide you how to access the restful API and decode the response :

public static void main(String[] args) throws Exception {
    if (args.length < 4) {
        System.out.println("Usage: <configPath> <query> <schemaPath> <statsPath>");
        System.exit(1);
    }
    // set request body in json format
    String jsonPayLoad = createParameters(args[0], args[1], args[2], args[3]).toString();
    HttpClient client = HttpClient.newBuilder().build();
    // create http request, set header and body content
    HttpRequest request =
            HttpRequest.newBuilder()
                    .uri(URI.create("http://localhost:8080/api/compilePlan"))
                    .setHeader("Content-Type", "application/json")
                    .POST(
                            HttpRequest.BodyPublishers.ofString(
                                    jsonPayLoad, StandardCharsets.UTF_8))
                    .build();
    // send request and get response
    HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
    String body = response.body();
    // parse response body as json
    JsonNode planNode = (new ObjectMapper()).readTree(body).get("graphPlan");
    // print result
    System.out.println(getPhysicalPlan(planNode));
    System.out.println(getResultSchemaYaml(planNode));
}

private static JsonNode createParameters(
        String configPath, String query, String schemaPath, String statsPath) throws Exception {
    Map<String, String> params =
            ImmutableMap.of(
                    "configPath",
                    configPath,
                    "query",
                    query,
                    "schemaYaml",
                    FileUtils.readFileToString(new File(schemaPath), StandardCharsets.UTF_8),
                    "statsJson",
                    FileUtils.readFileToString(new File(statsPath), StandardCharsets.UTF_8));
    return (new ObjectMapper()).valueToTree(params);
}

// get base64 string from json, convert it to physical bytes , then parse it to PhysicalPlan
private static GraphAlgebraPhysical.PhysicalPlan getPhysicalPlan(JsonNode planNode)
        throws Exception {
    String base64Str = planNode.get("physicalBytes").asText();
    byte[] bytes = java.util.Base64.getDecoder().decode(base64Str);
    return GraphAlgebraPhysical.PhysicalPlan.parseFrom(bytes);
}

// get result schema yaml from json
private static String getResultSchemaYaml(JsonNode planNode) {
    return planNode.get("resultSchemaYaml").asText();
}

Run the java sdk example with the following command:

java -cp ".:./libs/*" com.alibaba.graphscope.sdk.examples.TestGraphPlanner ./conf/gs_interactive_hiactor.yaml "Match (n) Return n;" ./conf/graph.yaml ./conf/modern_statistics.json

Graph Planner Interface by JNI and RESTful API¶

JNI API¶

Interface Overview¶

Getting Started¶

Step 1: Build the Project¶

Step 2: Locate and Extract the Package¶

Step 3: Run the Example Binary¶

Restful API¶

Getting Started¶

Step 1: Build the Project¶

Step 2: Locate and Extract the Package¶

Step 3: Start the Graph Planner RESTful Service¶

Step 4: Access the RESTful API by Curl¶

Step 4: Access the RESTful API by Java Sdk¶

Step 4: Access the RESTful API by `Curl`¶

Step 4: Access the RESTful API by `Java` Sdk¶