Analytical Engine API Reference

Analytical Engine

class gs::GrapeEngine

GrapeEngine is responsible to assemble and start four major components of analytical engine, including VineyardServer, GrapeInstance, Dispatcher and GraphScopeServer.

Public Functions

GrapeEngine(std::string host, int port)

Construct a new GrapeEngine object in service mode, listening at the host:port for incoming rpc requests.

Parameters
  • host:

  • port:

GrapeEngine(std::string dag_file)

Construct a new GrapeEngine object in job mode, processing the workflow defined in the dagfile.

Parameters
  • dag_file:

void Start()
void Stop()
int RunDAGFile()
class gs::GrapeInstance : public gs::Subscriber

MPI management.

This controller initials MPI communication world, assign a rank to each process. According to the assigned rank, determines whether this process runs as a coordinator or a worker. It also in charges of execute commands from coordinator.

Public Functions

GrapeInstance(const grape::CommSpec &comm_spec)
void Init(const std::string &vineyard_socket)
bl::result<std::shared_ptr<DispatchResult>> OnReceive(const CommandDetail &cmd) override
class gs::GSObject

GSObject is the base class of GraphScope object. Every object which is managed by ObjectManager should inherit GSObject.

Subclassed by gs::AppEntry, gs::IContextWrapper, gs::IFragmentWrapper, gs::Projector, gs::PropertyGraphUtils

Public Functions

GSObject(std::string id, ObjectType type)
~GSObject()
const std::string &id() const
ObjectType type() const
std::string ToString() const
class gs::ObjectManager

ObjectManager manages GSObject like fragment wrapper, loaded app and more.

Public Functions

bl::result<void> PutObject(std::shared_ptr<GSObject> obj)
bl::result<void> RemoveObject(const std::string &id)
bl::result<std::shared_ptr<GSObject>> GetObject(const std::string &id)
template<typename T>
bl::result<std::shared_ptr<T>> GetObject(const std::string &id)
bool HasObject(const std::string &id)
class gs::rpc::AnalyticalServer

AnalyticalServer is responsible for create and start the gRPC service.

Public Functions

AnalyticalServer(std::shared_ptr<Dispatcher> dispatcher, std::string host, int port)
void StartServer()
void StopServer()
struct gs::EngineConfig

EngineConfig contains configurations about the analytical engine, such as networkx features in enabled or not, vineyard socket, and vineyard rpc endpoint.

Public Functions

std::string ToJsonString() const

Public Members

std::string networkx
std::string vineyard_socket
std::string vineyard_rpc_endpoint
class gs::Dispatcher

The dispatcher broadcast commands to every worker using MPI.

Public Functions

Dispatcher(const grape::CommSpec &comm_spec)
void Start()
void Stop()
std::vector<DispatchResult> Dispatch(CommandDetail &cmd)
void Subscribe(std::shared_ptr<Subscriber> subscriber)
void SetCommand(const CommandDetail &cmd)
class gs::DispatchResult

DispatchResult wraps computation result, metadata and error message for python client.

Public Types

enum AggregatePolicy

Values:

enumerator kPickFirst
enumerator kPickFirstNonEmpty
enumerator kRequireConsistent
enumerator kConcat

Public Functions

DispatchResult() = default
DispatchResult(int worker_id, rpc::Code error_code = rpc::Code::OK)
int worker_id() const
void set_error(rpc::Code error_code, const std::string &message)
rpc::Code error_code() const
std::string message()
void set_graph_def(const rpc::graph::GraphDefPb &graph_def)

Set the graph metadata. The meta should be kept consistent among all workers.

Parameters
  • graph_def:

const rpc::graph::GraphDefPb &graph_def() const
rpc::graph::GraphDefPb &graph_def()
void set_data(const std::string &data, AggregatePolicy policy = AggregatePolicy::kRequireConsistent)
void set_data(const grape::InArchive &arc, AggregatePolicy policy = AggregatePolicy::kRequireConsistent)
const std::string &data()
AggregatePolicy aggregate_policy() const

Fragments in GraphScope

template<typename FRAG_T>
class FragmentWrapper

This is a fragment wrapper, which means a series of methods are provided to serialize/transform the data attached to the fragment. An AddColumn method is provided to add properties to create a new fragment from the original one.

Template Parameters
  • FRAG_T:

template<typename OID_T, typename VID_T, typename VDATA_T, typename EDATA_T>
class gs::ArrowProjectedFragment : public gs::ArrowProjectedFragmentBase, public vineyard::BareRegistered<ArrowProjectedFragment<OID_T, VID_T, VDATA_T, EDATA_T>>

This class represents the fragment projected from ArrowFragment. The fragment has no label and property.

Template Parameters
  • OID_T: OID type

  • VID_T: VID type

  • VDATA_T: The type of data attached with the vertex

  • EDATA_T: The type of data attached with the edge

Public Types

using oid_t = OID_T
using vid_t = VID_T
using internal_oid_t = typename vineyard::InternalType<oid_t>::type
using eid_t = vineyard::property_graph_types::EID_TYPE
using vertex_range_t = grape::VertexRange<vid_t>
using vertex_t = grape::Vertex<vid_t>
using nbr_t = arrow_projected_fragment_impl::Nbr<vid_t, eid_t, EDATA_T>
using nbr_unit_t = vineyard::property_graph_utils::NbrUnit<vid_t, eid_t>
using adj_list_t = arrow_projected_fragment_impl::AdjList<vid_t, eid_t, EDATA_T>
using const_adj_list_t = arrow_projected_fragment_impl::AdjList<vid_t, eid_t, EDATA_T>
using vertex_map_t = ArrowProjectedVertexMap<internal_oid_t, vid_t>
using label_id_t = vineyard::property_graph_types::LABEL_ID_TYPE
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using vdata_t = VDATA_T
using edata_t = EDATA_T
using property_graph_t = vineyard::ArrowFragment<oid_t, vid_t>
using vid_array_t = typename vineyard::ConvertToArrowType<vid_t>::ArrayType
using eid_array_t = typename vineyard::ConvertToArrowType<eid_t>::ArrayType
template<typename DATA_T>
using vertex_array_t = grape::VertexArray<DATA_T, vid_t>

Public Functions

~ArrowProjectedFragment()
void Construct(const vineyard::ObjectMeta &meta) override
void PrepareToRunApp(grape::MessageStrategy strategy, bool need_split_edges)
fid_t fid() const
fid_t fnum() const
vertex_range_t Vertices() const
vertex_range_t InnerVertices() const
vertex_range_t OuterVertices() const
vertex_range_t OuterVertices(fid_t fid) const
const std::vector<vertex_t> &MirrorVertices(fid_t fid) const
bool GetVertex(const oid_t &oid, vertex_t &v) const
oid_t GetId(const vertex_t &v) const
fid_t GetFragId(const vertex_t &v) const
arrow_projected_fragment_impl::TypedArray<VDATA_T>::value_type GetData(const vertex_t &v) const
bool Gid2Vertex(const vid_t &gid, vertex_t &v) const
vid_t Vertex2Gid(const vertex_t &v) const
vid_t GetInnerVerticesNum() const
vid_t GetOuterVerticesNum() const
vid_t GetVerticesNum() const
size_t GetEdgeNum() const
size_t GetTotalVerticesNum() const
bool IsInnerVertex(const vertex_t &v) const
bool IsOuterVertex(const vertex_t &v) const
bool GetInnerVertex(const oid_t &oid, vertex_t &v) const
bool GetOuterVertex(const oid_t &oid, vertex_t &v) const
oid_t GetInnerVertexId(const vertex_t &v) const
oid_t GetOuterVertexId(const vertex_t &v) const
oid_t Gid2Oid(const vid_t &gid) const
bool Oid2Gid(const oid_t &oid, vid_t &gid) const
bool InnerVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
bool OuterVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
vid_t GetOuterVertexGid(const vertex_t &v) const
vid_t GetInnerVertexGid(const vertex_t &v) const
adj_list_t GetIncomingAdjList(const vertex_t &v) const
adj_list_t GetOutgoingAdjList(const vertex_t &v) const
adj_list_t GetIncomingInnerVertexAdjList(const vertex_t &v) const
adj_list_t GetOutgoingInnerVertexAdjList(const vertex_t &v) const
adj_list_t GetIncomingOuterVertexAdjList(const vertex_t &v) const
adj_list_t GetOutgoingOuterVertexAdjList(const vertex_t &v) const
adj_list_t GetIncomingAdjList(const vertex_t &v, fid_t src_fid) const
adj_list_t GetOutgoingAdjList(const vertex_t &v, fid_t dst_fid) const
int GetLocalOutDegree(const vertex_t &v) const
int GetLocalInDegree(const vertex_t &v) const
grape::DestList IEDests(const vertex_t &v) const
grape::DestList OEDests(const vertex_t &v) const
grape::DestList IOEDests(const vertex_t &v) const

Public Static Functions

std::shared_ptr<vineyard::Object> Create()
std::shared_ptr<ArrowProjectedFragment<oid_t, vid_t, vdata_t, edata_t>> Project(std::shared_ptr<vineyard::ArrowFragment<oid_t, vid_t>> fragment, const std::string &v_label_str, const std::string &v_prop_str, const std::string &e_label_str, const std::string &e_prop_str)

Public Static Attributes

constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename OID_T, typename VID_T>
class gs::AppendOnlyArrowFragment : public vineyard::Registered<AppendOnlyArrowFragment<OID_T, VID_T>>

A labeled fragment can be modified vertices and edges in append fashion. The data in the fragment are stored in two parts. The first part is initial data which will be stored in the vineyard. The second part is later coming data (append data) which will be located in the engine’s memory space.

Template Parameters
  • OID_T:

  • VID_T:

Public Types

using oid_t = OID_T
using vid_t = VID_T
using internal_oid_t = typename vineyard::InternalType<oid_t>::type
using eid_t = vineyard::property_graph_types::EID_TYPE
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using label_id_t = vineyard::property_graph_types::LABEL_ID_TYPE
using vertex_range_t = grape::VertexRange<vid_t>
using nbr_t = vineyard::property_graph_utils::OffsetNbr<vid_t, eid_t>
using nbr_unit_t = vineyard::property_graph_utils::NbrUnit<vid_t, eid_t>
using adj_list_t = vineyard::property_graph_utils::OffsetAdjList<vid_t, eid_t>
using extra_nbr_t = append_only_fragment_impl::ExtraNbr<vid_t, eid_t>
using extra_adj_list_t = append_only_fragment_impl::ExtraAdjList<vid_t, eid_t>
using vertex_map_t = vineyard::ArrowVertexMap<internal_oid_t, vid_t>
using extra_vertex_map_t = ExtraVertexMap<oid_t, vid_t>
using vertex_t = grape::Vertex<vid_t>
using vid_array_t = typename vineyard::ConvertToArrowType<vid_t>::ArrayType
using eid_array_t = typename vineyard::ConvertToArrowType<eid_t>::ArrayType
template<typename DATA_T>
using vertex_array_t = grape::VertexArray<DATA_T, vid_t>

Public Functions

void Construct(const vineyard::ObjectMeta &meta) override
fid_t fid() const
fid_t fnum() const
label_id_t vertex_label_num() const
label_id_t vertex_label(const vertex_t &v) const
int64_t vertex_offset(const vertex_t &v) const
label_id_t edge_label_num() const
std::shared_ptr<arrow::Table> vertex_data_table(label_id_t i) const
std::shared_ptr<arrow::Table> edge_data_table(label_id_t i) const
vertex_range_t Vertices(label_id_t label_id) const
vertex_range_t InnerVertices(label_id_t label_id) const
vertex_range_t OuterVertices(label_id_t label_id) const
bool GetVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
oid_t GetId(const vertex_t &v) const
fid_t GetFragId(const vertex_t &u) const
size_t GetTotalNodesNum() const
template<typename T>
T GetData(const vertex_t &v, prop_id_t prop_id) const
bool HasChild(const vertex_t &v, label_id_t e_label) const
bool HasParent(const vertex_t &v, label_id_t e_label) const
int GetLocalOutDegree(const vertex_t &v, label_id_t e_label) const
int GetLocalInDegree(const vertex_t &v, label_id_t e_label) const
bool Gid2Vertex(const vid_t &gid, vertex_t &v) const
vid_t Vertex2Gid(const vertex_t &v) const
vid_t GetInnerVerticesNum(label_id_t label_id) const
vid_t GetOuterVerticesNum(label_id_t label_id) const
bool IsInnerVertex(const vertex_t &v) const
bool IsOuterVertex(const vertex_t &v) const
bool GetInnerVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
bool GetOuterVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
oid_t GetInnerVertexId(const vertex_t &v) const
oid_t GetOuterVertexId(const vertex_t &v) const
oid_t Gid2Oid(const vid_t &gid) const
bool Oid2Gid(label_id_t label, const oid_t &oid, vid_t &gid) const
bool InnerVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
bool OuterVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
vid_t GetOuterVertexGid(const vertex_t &v) const
vid_t GetInnerVertexGid(const vertex_t &v) const
adj_list_t GetIncomingAdjList(const vertex_t &v, label_id_t e_label) const
adj_list_t GetOutgoingAdjList(const vertex_t &v, label_id_t e_label) const
extra_adj_list_t GetExtraOutgoingAdjList(const vertex_t &v, label_id_t e_label) const
extra_adj_list_t GetExtraIncomingAdjList(const vertex_t &v, label_id_t e_label) const
grape::DestList IEDests(const vertex_t &v, label_id_t e_label) const
grape::DestList OEDests(const vertex_t &v, label_id_t e_label) const
grape::DestList IOEDests(const vertex_t &v, label_id_t e_label) const
std::shared_ptr<vertex_map_t> GetVertexMap()
void PrepareToRunApp(grape::MessageStrategy strategy, bool need_split_edges)
std::shared_ptr<extra_vertex_map_t> GetExtraVertexMap()
std::shared_ptr<AppendOnlyArrowTable> extra_edge_data_table(label_id_t i) const

Public Static Functions

std::shared_ptr<vineyard::Object> Create()
template<typename OID_T, typename VID_T>
class gs::ArrowProjectedVertexMap : public vineyard::Registered<ArrowProjectedVertexMap<OID_T, VID_T>>

This class represents the mapping between oid and vid.

Template Parameters
  • OID_T: OID type

  • VID_T: VID type

Public Functions

void Construct(const vineyard::ObjectMeta &meta)
bool GetOid(vid_t gid, oid_t &oid) const
bool GetGid(fid_t fid, oid_t oid, vid_t &gid) const
bool GetGid(oid_t oid, vid_t &gid) const
size_t GetTotalVerticesNum() const

Public Static Functions

std::shared_ptr<vineyard::Object> Create()
std::shared_ptr<ArrowProjectedVertexMap<OID_T, VID_T>> Project(std::shared_ptr<vineyard::ArrowVertexMap<OID_T, VID_T>> vm, label_id_t v_label)
template<typename OID_T = vineyard::property_graph_types::OID_TYPE, typename VID_T = vineyard::property_graph_types::VID_TYPE>
class gs::ArrowFragmentLoader

This loader can load a ArrowFragment from the data source including local file, oss, numpy, pandas and vineyard.

Template Parameters
  • OID_T: OID type

  • VID_T: VID type

Public Functions

ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, const std::vector<std::string> &efiles, const std::vector<std::string> &vfiles, bool directed = true)
ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, std::shared_ptr<detail::Graph> graph_info)
~ArrowFragmentLoader() = default
boost::leaf::result<std::vector<std::shared_ptr<arrow::Table>>> LoadVertexTables()
boost::leaf::result<std::vector<std::vector<std::shared_ptr<arrow::Table>>>> LoadEdgeTables()
boost::leaf::result<vineyard::ObjectID> AddLabelsToGraph(vineyard::ObjectID frag_id)
boost::leaf::result<vineyard::ObjectID> addVerticesAndEdges(vineyard::ObjectID frag_id)
boost::leaf::result<vineyard::ObjectID> addVertices(vineyard::ObjectID frag_id)
boost::leaf::result<vineyard::ObjectID> addEdges(vineyard::ObjectID frag_id)
boost::leaf::result<vineyard::ObjectID> LoadFragment()
boost::leaf::result<vineyard::ObjectID> AddLabelsToGraphAsFragmentGroup(vineyard::ObjectID frag_id)
boost::leaf::result<vineyard::ObjectID> LoadFragmentAsFragmentGroup()
boost::leaf::result<partitioner_t> initPartitioner()
template<typename OID_T, typename VID_T>
class gs::ArrowFragmentAppender

ArrowFragmentAppender is a utility to modify AppendOnlyArrowFragment in append fashion.

Template Parameters
  • OID_T:

  • VID_T:

Public Functions

ArrowFragmentAppender(grape::CommSpec &comm_spec, std::shared_ptr<AppendOnlyArrowFragment<OID_T, VID_T>> fragment)
bl::result<int64_t> ExtendFragment(std::vector<std::vector<std::string>> &vertex_messages, std::vector<std::vector<std::string>> &edge_messages, bool header_row, char delimiter, bool directed)

Only should be invoked on Coordinator process

Parameters
  • vertex_messages:

  • edge_messages:

  • directed:

template<typename OID_T, typename VID_T>
class gs::AppendOnlyArrowFragmentBuilder

Subclassed by gs::BasicAppendOnlyArrowFragmentBuilder< OID_T, VID_T >

Public Functions

AppendOnlyArrowFragmentBuilder(vineyard::Client &client)
void set_fid(fid_t fid)
void set_fnum(fid_t fnum)
void set_directed(bool directed)
void set_label_num(label_id_t vertex_label_num, label_id_t edge_label_num)
void set_ivnums(const vineyard::Array<vid_t> &ivnums)
void set_ovnums(const vineyard::Array<vid_t> &ovnums)
void set_tvnums(const vineyard::Array<vid_t> &tvnums)
void set_vertex_table(label_id_t label, std::shared_ptr<vineyard::Table> table)
void set_ovgid_list(label_id_t label, std::shared_ptr<vineyard::NumericArray<vid_t>> ovgid_list)
void set_ovg2l_map(label_id_t label, std::shared_ptr<vineyard::Hashmap<vid_t, vid_t>> ovg2l_map)
void set_edge_table(label_id_t label, std::shared_ptr<vineyard::Table> table)
void set_in_edge_list(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::FixedSizeBinaryArray> in_edge_list)
void set_out_edge_list(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::FixedSizeBinaryArray> out_edge_list)
void set_in_edge_offsets(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::NumericArray<int64_t>> in_edge_offsets)
void set_out_edge_offsets(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::NumericArray<int64_t>> out_edge_offsets)
void set_vertex_map(std::shared_ptr<vertex_map_t> vm_ptr)
std::shared_ptr<vineyard::Object> _Seal(vineyard::Client &client) override
template<typename OID_T = vineyard::property_graph_types::OID_TYPE, typename VID_T = vineyard::property_graph_types::VID_TYPE>
class gs::AppendOnlyArrowFragmentLoader

AppendOnlyArrowFragmentLoader loads AppendOnlyArrowFragment from local files.

Template Parameters
  • OID_T:

  • VID_T:

Public Functions

AppendOnlyArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, label_id_t vertex_label_num, label_id_t edge_label_num, std::string efile, std::string vfile, bool directed = true)
~AppendOnlyArrowFragmentLoader() = default
bl::result<vineyard::ObjectID> LoadFragment()

Analytical Apps

template<typename FRAG_T, typename CONTEXT_T>
class gs::AppBase

AppBase is the base class for applications.

It contains an grape::DefaultMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

Template Parameters
  • FRAG_T: Fragment class

  • CONTEXT_T: App context class

Public Types

using message_manager_t = grape::DefaultMessageManager

Public Functions

AppBase() = default
~AppBase() = default
void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

Public Static Attributes

constexpr bool need_split_edges = false
constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kOnlyOut
template<typename FRAG_T, typename CONTEXT_T>
class gs::PropertyAppBase

PropertyAppBase is the base class for applications on property graphs.

It contains an PropertyMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

Template Parameters
  • FRAG_T:

  • CONTEXT_T:

Public Types

using message_manager_t = PropertyMessageManager

Public Functions

PropertyAppBase() = default
~PropertyAppBase() = default
void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

Public Static Attributes

constexpr bool need_split_edges = false
constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename FRAG_T, typename CONTEXT_T>
class gs::PropertyAutoAppBase

PropertyAutoAppBase is the base class for applications on property graph, with an PropertyAutoMessageManager to process messages during execution of application. Apps derived from this class do not need to process messages explicitly.

Template Parameters
  • FRAG_T:

  • CONTEXT_T:

Public Functions

PropertyAutoAppBase() = default
~PropertyAutoAppBase() = default
void PEval(const FRAG_T &graph, CONTEXT_T &context) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

void IncEval(const FRAG_T &graph, CONTEXT_T &context) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph:

  • context:

  • messages:

Public Static Attributes

constexpr bool need_split_edges = false
constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename APP_T>
class gs::DefaultWorker

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = grape::DefaultMessageManager

Public Functions

DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
template<typename APP_T>
class gs::ParallelPropertyWorker

A parallel worker for labeled fragment.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = ParallelPropertyMessageManager

Public Functions

ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~ParallelPropertyWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
template<typename APP_T>
class gs::PropertyAutoWorker

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>

Public Functions

PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~PropertyAutoWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
class gs::AppEntry : public gs::GSObject

AppEntry is a class manages an application.

AppEntry holds the a group of function pointers to manipulate the AppFrame, such as gs::CreateWorker, DeleteWorker and Query. The method Init must be called to load the library before use.

Public Functions

AppEntry(std::string id, std::string lib_path)
bl::result<void> Init()
bl::result<std::shared_ptr<void>> CreateWorker(const std::shared_ptr<void> &fragment, const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &spec)
bl::result<std::shared_ptr<IContextWrapper>> Query(void *worker_handler, const rpc::QueryArgs &query_args, const std::string &ctx_name, std::shared_ptr<IFragmentWrapper> &frag_wrapper)
template<class FunctionType>
struct ArgsNum
template<typename T>
struct ArgsUnpacker

ArgsUnpacker is an util to unpack the Any type of proto to C++ datatype.

Template Parameters
  • T: The destination type

template<std::size_t index, class FunctionType>
struct ArgTypeAt

Analytical Workers

template<typename APP_T>
class gs::DefaultWorker

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = grape::DefaultMessageManager

Public Functions

DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
template<typename APP_T>
class gs::DefaultPropertyWorker

A Worker manages the computation cycle. DefaultPropertyWorker is a kind of worker for apps derived from AppBase.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyMessageManager

Public Functions

DefaultPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultPropertyWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
template<typename APP_T>
class gs::ParallelPropertyWorker

A parallel worker for labeled fragment.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = ParallelPropertyMessageManager

Public Functions

ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~ParallelPropertyWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)
template<typename APP_T>
class gs::PropertyAutoWorker

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

Template Parameters
  • APP_T:

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>

Public Functions

PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~PropertyAutoWorker() = default
void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
void Finalize()
template<class ...Args>
void Query(Args&&... args)
std::shared_ptr<context_t> GetContext()
void Output(std::ostream &os)

App’s Context

template<typename CTX_T, typename = void>
struct CtxWrapperBuilder

This is a utility to build kinds of ContextWrapper instances.

Template Parameters
  • CTX_T:

template<typename FRAG_T>
class gs::VertexPropertyContext : public ContextBase

VertexPropertyContext can hold any number of columns. The context is designed for labeled fragment - ArrowFragment. Compared with LabeledVertexDataContext, the data type and column count can be determined at runtime.

Template Parameters
  • FRAG_T: The fragment class (non-labeled fragment only)

Public Types

using fragment_t = FRAG_T
using vertex_t = typename fragment_t::vertex_t
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using oid_t = typename fragment_t::oid_t

Public Functions

VertexPropertyContext(const fragment_t &fragment)
const fragment_t &fragment()
int64_t add_column(const std::string &name, ContextDataType type)
std::shared_ptr<IColumn> get_column(int64_t index)
std::shared_ptr<IColumn> get_column(const std::string &name)
template<typename DATA_T>
std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(int64_t index)
template<typename DATA_T>
std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(const std::string &name)
std::vector<std::shared_ptr<IColumn>> &vertex_properties()
const std::map<std::string, std::shared_ptr<IColumn>> &properties_map()
template<typename FRAG_T, typename DATA_T>
class gs::VertexDataContextWrapper : public gs::IVertexDataContextWrapper

This is the wrapper class for VertexDataContext. A series of methods are provided to transform the data hold by the context.

Template Parameters
  • FRAG_T: The fragment class (Non-labeled fragment only)

  • DATA_T: The Data type hold by context

Public Functions

VertexDataContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)
std::string context_type() override
std::shared_ptr<IFragmentWrapper> fragment_wrapper() override
bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, const Selector &selector, const std::pair<std::string, std::string> &range) override
bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override
bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, const Selector &selector, const std::pair<std::string, std::string> &range) override
bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override
bl::result<std::vector<std::pair<std::string, std::shared_ptr<arrow::Array>>>> ToArrowArrays(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors) override
template<typename FRAG_T, typename DATA_T>
class gs::TensorContext : public ContextBase

TensorContext is designed for holding a bunch of computation results. The TensorContext should be used if the number of elements are not related to the number of the vertex.

Template Parameters
  • FRAG_T:

  • DATA_T:

Public Types

using data_t = DATA_T

Public Functions

TensorContext(const fragment_t &fragment)
const fragment_t &fragment()
void assign(const std::vector<data_t> &data, const std::vector<size_t> &shape)
void assign(const data_t &data)
void set_shape(std::vector<std::size_t> shape)
std::vector<size_t> shape() const
tensor_t &tensor()
template<typename FRAG_T, typename DATA_T, typename = void>
class gs::TensorContextWrapper : public gs::ITensorContextWrapper

TensorContextWrapper is the wrapper class for TensorContext.

Template Parameters
  • FRAG_T:

  • DATA_T:

Public Functions

TensorContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)
std::string context_type() override
std::shared_ptr<IFragmentWrapper> fragment_wrapper() override
bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, uint32_t axis) override
bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec) override
bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, uint32_t axis) override
bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client) override