Simplifying Complex Graph Loading with Jupyter Notebook

Schema construction and graph data loading are usually the complicated steps in graph computing processes. Currently, GraphScope has released a graphscope-notebook plugin which through an interactive way help users complete the graph loading in the Jupyterlab environment. This article will provide a detailed introduction to the use of this plugin, and users can try it in the Playground environment.

Background

For any graph computing product, the loading of graph data is often the first and most important step, as well as a very complicated step, mainly due to the complexity of the graph data itself. Therefore, in order to improve the loading experience, GraphScope has built-in various datasets. For example, for the TinkerPop Modern Graph, users just need one statement to complete the loading operation:

>>> from graphscope.dataset import load_modern_graph
>>> modern_graph = load_modern_graph()

However, for the user’s dataset, the loading process needs to define a very long code, we use ogbn-mag this property graph as an example:

This graph has four kinds of vertices, labeled as paper, author, institution and field_of_study. There are four kinds of edges connecting them, each kind of edges has a label and specifies the vertex labels for its two ends. For example, cites edges connect two vertices labeled paper. Another example is writes, it requires the source vertex is labeled author and the destination is a paper vertex. All the vertices and edges may have properties. e.g., paper vertices have properties like features, publish year, subject label, etc.

Usually, each type of vertex(edge) corresponds to a csv file, which can be downloaded from here.

$ tree
├── author_affiliated_with_institution.csv
├── author.csv
├── author_writes_paper.csv
├── field_of_study.csv
├── institution.csv
├── paper_cites_paper.csv
├── paper.csv
└── paper_has_topic_field_of_study.csv

Finally, for the user, the actual loading code is as follows:

We can see that even for such a flexible language as Python, the schema definition is very complicated for the above-mentioned property graph containing 4 labels, not to mention hundreds of labels of vertices and edges. Even each vertex(edge) may have thousands of properties.

Therefore, in order to reduce the complexity and error rate of the loading process, GraphScope has developed a graphscope-notebook plugin, which can help GraphScope to complete the loading process of complex graph data interactively in the Jupyterlab environment. Currently, the plugin has been deployed in the GraphScope Playground environment, and everyone is welcome to try it out. Next, this article will use this above-mentioned property graph as an example to introduce in detail how to load graph interactively in the Jupyterlab environment.

Plugin Installation

The plugin requires the following conditions:

• JupyterLab >= 3.0
• GraphScope >= 0.12.0

You can install the plugin with the following command:

pip3 install graphscope-notebook

It is worth noting that after the plugin installation is complete, you need to restart Jupyterlab. Finally, if the left sidebar displays as following, it means that the installation is successful; Or you can find it in GraphScope Community to report problems encountered.

Using the Plugin

First, we run the following code in the Jupyterlab to create a GraphScope Session.

>>> import graphscope
>> graphscope.set_option(show_log=True)
>>> sess = graphscope.session(cluster_type="hosts")

After the Session is created, we can monitor the resource in the left resource panel. Click the “+” button to open the interactive page on the right side of the notebook:

1. Create Vertex

Next, we can complete the graph data construction according to the prompts on the interactive page. Take the “paper” type of vertex as an example. The page for creating points is as follows. After the information is filled in, click “Create Vertex” to complete the construction of the point.

Next, we can complete the construction of the graph data according to the prompts on the interactive page. Taking the “paper” type of vertex as an example. After filling in the information, click “Create Vertex” to complete the vertex construction.

The explanations for each field are as follows:

2. Create Edge

Similar to “Create Vertex”, take the “cites” type of edge as an example (paper -> cites -> paper). After the information is filled in, click “Create Edge” to complete the construction of the edge.

3. Define Graph-Related Information

After building all types of vertices/edges, we set the graph-related information:

4. Generate Code and Load Graph

After all the information above is filled in, select one “Notebook Cell” with the mouse, and click the “Generate Code” button to generate the corresponding graph loading code. Run this cell to finish the data loading process, and monitor the graph resource in the left resource panel:

So far, we have successfully used the graphscope-notebook plugin to complete the graph data loading process. Next, we can refer to the documentation to play with GraphScope.

Conclusion

The graphscope-notebook jupterlab plugin currently has the functions of 1) monitoring GraphScope runtime resources; 2) loading graph through an interactive way to reduce the complexity during graph loading process. In addition, we also plan to add visualization analysis of graph data in subsequent versions. Stay tuned!