Analytical Engine API Reference

Analytical Engine

class gs::GrapeEngine

GrapeEngine is responsible to assemble and start four major components of analytical engine, including VineyardServer, GrapeInstance, Dispatcher and GraphScopeServer.

Public Functions

inline GrapeEngine(std::string host, int port)

Construct a new GrapeEngine object in service mode, listening at the host:port for incoming rpc requests.

Parameters
  • host

  • port

inline explicit GrapeEngine(std::string dag_file)

Construct a new GrapeEngine object in job mode, processing the workflow defined in the dagfile.

Parameters

dag_file

inline void Start()
inline void Stop()
inline int RunDAGFile()
class gs::GrapeInstance : public gs::Subscriber

MPI management.

This controller initials MPI communication world, assign a rank to each process. According to the assigned rank, determines whether this process runs as a coordinator or a worker. It also in charges of execute commands from coordinator.

Public Functions

explicit GrapeInstance(const grape::CommSpec &comm_spec)
void Init(const std::string &vineyard_socket)
virtual bl::result<std::shared_ptr<DispatchResult>> OnReceive(const CommandDetail &cmd) override
class gs::GSObject

GSObject is the base class of GraphScope object. Every object which is managed by ObjectManager should inherit GSObject.

Subclassed by gs::AppEntry, gs::IContextWrapper, gs::IFragmentWrapper, gs::Projector, gs::PropertyGraphUtils

Public Functions

inline explicit GSObject(std::string id, ObjectType type)
inline virtual ~GSObject()
inline const std::string &id() const
inline ObjectType type() const
inline virtual std::string ToString() const
class gs::ObjectManager

ObjectManager manages GSObject like fragment wrapper, loaded app and more.

Public Functions

inline bl::result<void> PutObject(std::shared_ptr<GSObject> obj)
inline bl::result<void> RemoveObject(const std::string &id)
inline bl::result<std::shared_ptr<GSObject>> GetObject(const std::string &id)
template<typename T>
inline bl::result<std::shared_ptr<T>> GetObject(const std::string &id)
inline bool HasObject(const std::string &id)
class gs::rpc::AnalyticalServer

AnalyticalServer is responsible for create and start the gRPC service.

Public Functions

inline AnalyticalServer(std::shared_ptr<Dispatcher> dispatcher, std::string host, int port)
void StartServer()
void StopServer()
struct gs::EngineConfig

EngineConfig contains configurations about the analytical engine, such as networkx features in enabled or not, vineyard socket, and vineyard rpc endpoint.

Public Functions

inline std::string ToJsonString() const

Public Members

std::string networkx
std::string vineyard_socket
std::string vineyard_rpc_endpoint
class gs::Dispatcher

The dispatcher broadcast commands to every worker using MPI.

Public Functions

explicit Dispatcher(const grape::CommSpec &comm_spec)
void Start()
void Stop()
std::vector<DispatchResult> Dispatch(CommandDetail &cmd)
void Subscribe(std::shared_ptr<Subscriber> subscriber)
void SetCommand(const CommandDetail &cmd)
class gs::DispatchResult

DispatchResult wraps computation result, metadata and error message for python client.

Public Types

enum AggregatePolicy

Values:

enumerator kPickFirst
enumerator kPickFirstNonEmpty
enumerator kRequireConsistent
enumerator kConcat
enumerator kPickFirstNonEmptyGraphDef
enumerator kMergeGraphDef

Public Functions

DispatchResult() = default
inline explicit DispatchResult(int worker_id, rpc::Code error_code = rpc::Code::OK)
inline int worker_id() const
inline void set_error(rpc::Code error_code, const std::string &message)
inline rpc::Code error_code() const
inline std::string message()
inline void set_graph_def(const rpc::graph::GraphDefPb &graph_def, AggregatePolicy policy = AggregatePolicy::kMergeGraphDef)

Set the graph metadata. The meta should be kept consistent among all workers.

Parameters

graph_def

inline const rpc::graph::GraphDefPb &graph_def() const
inline rpc::graph::GraphDefPb &graph_def()
inline void set_data(const std::string &data, AggregatePolicy policy = AggregatePolicy::kRequireConsistent)
inline void set_data(const grape::InArchive &arc, AggregatePolicy policy = AggregatePolicy::kRequireConsistent)
inline const std::string &data()
inline AggregatePolicy aggregate_policy() const

Fragments in GraphScope

template<typename FRAG_T>
class FragmentWrapper

This is a fragment wrapper, which means a series of methods are provided to serialize/transform the data attached to the fragment. An AddColumn method is provided to add properties to create a new fragment from the original one.

tparam FRAG_T

template<typename OID_T, typename VID_T, typename VDATA_T, typename EDATA_T>
class gs::ArrowProjectedFragment : public gs::ArrowProjectedFragmentBase, public vineyard::BareRegistered<ArrowProjectedFragment<OID_T, VID_T, VDATA_T, EDATA_T>>

This class represents the fragment projected from ArrowFragment which contains only one vertex label and edge label. The fragment has no label and property.

tparam OID_T

OID type

tparam VID_T

VID type

tparam VDATA_T

The type of data attached with the vertex

tparam EDATA_T

The type of data attached with the edge

Public Types

using oid_t = OID_T
using vid_t = VID_T
using internal_oid_t = typename vineyard::InternalType<oid_t>::type
using eid_t = vineyard::property_graph_types::EID_TYPE
using vertex_range_t = grape::VertexRange<vid_t>
using vertex_t = grape::Vertex<vid_t>
using nbr_t = arrow_projected_fragment_impl::Nbr<vid_t, eid_t, EDATA_T>
using nbr_unit_t = vineyard::property_graph_utils::NbrUnit<vid_t, eid_t>
using adj_list_t = arrow_projected_fragment_impl::AdjList<vid_t, eid_t, EDATA_T>
using const_adj_list_t = arrow_projected_fragment_impl::AdjList<vid_t, eid_t, EDATA_T>
using vertex_map_t = ArrowProjectedVertexMap<internal_oid_t, vid_t>
using label_id_t = vineyard::property_graph_types::LABEL_ID_TYPE
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using vdata_t = VDATA_T
using edata_t = EDATA_T
using property_graph_t = vineyard::ArrowFragment<oid_t, vid_t>
using vid_array_t = typename vineyard::ConvertToArrowType<vid_t>::ArrayType
using eid_array_t = typename vineyard::ConvertToArrowType<eid_t>::ArrayType
template<typename DATA_T>
using vertex_array_t = grape::VertexArray<DATA_T, vid_t>

Public Functions

inline ~ArrowProjectedFragment()
inline void Construct(const vineyard::ObjectMeta &meta) override
inline void PrepareToRunApp(grape::MessageStrategy strategy, bool need_split_edges)
inline fid_t fid() const
inline fid_t fnum() const
inline vertex_range_t Vertices() const
inline vertex_range_t InnerVertices() const
inline vertex_range_t OuterVertices() const
inline vertex_range_t OuterVertices(fid_t fid) const
inline const std::vector<vertex_t> &MirrorVertices(fid_t fid) const
inline bool GetVertex(const oid_t &oid, vertex_t &v) const
inline oid_t GetId(const vertex_t &v) const
inline fid_t GetFragId(const vertex_t &v) const
inline arrow_projected_fragment_impl::TypedArray<VDATA_T>::value_type GetData(const vertex_t &v) const
inline bool Gid2Vertex(const vid_t &gid, vertex_t &v) const
inline vid_t Vertex2Gid(const vertex_t &v) const
inline vid_t GetInnerVerticesNum() const
inline vid_t GetOuterVerticesNum() const
inline vid_t GetVerticesNum() const
inline size_t GetEdgeNum() const
inline size_t GetTotalVerticesNum() const
inline bool IsInnerVertex(const vertex_t &v) const
inline bool IsOuterVertex(const vertex_t &v) const
inline bool GetInnerVertex(const oid_t &oid, vertex_t &v) const
inline bool GetOuterVertex(const oid_t &oid, vertex_t &v) const
inline oid_t GetInnerVertexId(const vertex_t &v) const
inline oid_t GetOuterVertexId(const vertex_t &v) const
inline oid_t Gid2Oid(const vid_t &gid) const
inline bool Oid2Gid(const oid_t &oid, vid_t &gid) const
inline vid_t Oid2Gid(const oid_t &oid) const
inline bool InnerVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
inline bool OuterVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
inline vid_t GetOuterVertexGid(const vertex_t &v) const
inline vid_t GetInnerVertexGid(const vertex_t &v) const
inline adj_list_t GetIncomingAdjList(const vertex_t &v) const
inline adj_list_t GetOutgoingAdjList(const vertex_t &v) const
inline adj_list_t GetIncomingInnerVertexAdjList(const vertex_t &v) const
inline adj_list_t GetOutgoingInnerVertexAdjList(const vertex_t &v) const
inline adj_list_t GetIncomingOuterVertexAdjList(const vertex_t &v) const
inline adj_list_t GetOutgoingOuterVertexAdjList(const vertex_t &v) const
inline adj_list_t GetIncomingAdjList(const vertex_t &v, fid_t src_fid) const
inline adj_list_t GetOutgoingAdjList(const vertex_t &v, fid_t dst_fid) const
inline int GetLocalOutDegree(const vertex_t &v) const
inline int GetLocalInDegree(const vertex_t &v) const
inline grape::DestList IEDests(const vertex_t &v) const
inline grape::DestList OEDests(const vertex_t &v) const
inline grape::DestList IOEDests(const vertex_t &v) const
inline bool directed() const

Public Static Functions

static inline std::shared_ptr<vineyard::Object> Create()
static inline std::shared_ptr<ArrowProjectedFragment<oid_t, vid_t, vdata_t, edata_t>> Project(std::shared_ptr<vineyard::ArrowFragment<oid_t, vid_t>> fragment, const std::string &v_label_str, const std::string &v_prop_str, const std::string &e_label_str, const std::string &e_prop_str)

Public Static Attributes

static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename OID_T, typename VID_T>
class gs::AppendOnlyArrowFragment : public vineyard::Registered<AppendOnlyArrowFragment<OID_T, VID_T>>

A labeled fragment can be modified vertices and edges in append fashion. The data in the fragment are stored in two parts. The first part is initial data which will be stored in the vineyard. The second part is later coming data (append data) which will be located in the engine’s memory space.

tparam OID_T

tparam VID_T

Public Types

using oid_t = OID_T
using vid_t = VID_T
using internal_oid_t = typename vineyard::InternalType<oid_t>::type
using eid_t = vineyard::property_graph_types::EID_TYPE
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using label_id_t = vineyard::property_graph_types::LABEL_ID_TYPE
using vertex_range_t = grape::VertexRange<vid_t>
using nbr_t = vineyard::property_graph_utils::OffsetNbr<vid_t, eid_t>
using nbr_unit_t = vineyard::property_graph_utils::NbrUnit<vid_t, eid_t>
using adj_list_t = vineyard::property_graph_utils::OffsetAdjList<vid_t, eid_t>
using extra_nbr_t = append_only_fragment_impl::ExtraNbr<vid_t, eid_t>
using extra_adj_list_t = append_only_fragment_impl::ExtraAdjList<vid_t, eid_t>
using vertex_map_t = vineyard::ArrowVertexMap<internal_oid_t, vid_t>
using extra_vertex_map_t = ExtraVertexMap<oid_t, vid_t>
using vertex_t = grape::Vertex<vid_t>
using vid_array_t = typename vineyard::ConvertToArrowType<vid_t>::ArrayType
using eid_array_t = typename vineyard::ConvertToArrowType<eid_t>::ArrayType
template<typename DATA_T>
using vertex_array_t = grape::VertexArray<DATA_T, vid_t>

Public Functions

inline void Construct(const vineyard::ObjectMeta &meta) override
inline fid_t fid() const
inline fid_t fnum() const
inline label_id_t vertex_label_num() const
inline label_id_t vertex_label(const vertex_t &v) const
inline int64_t vertex_offset(const vertex_t &v) const
inline label_id_t edge_label_num() const
inline std::shared_ptr<arrow::Table> vertex_data_table(label_id_t i) const
inline std::shared_ptr<arrow::Table> edge_data_table(label_id_t i) const
inline vertex_range_t Vertices(label_id_t label_id) const
inline vertex_range_t InnerVertices(label_id_t label_id) const
inline vertex_range_t OuterVertices(label_id_t label_id) const
inline bool GetVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
inline oid_t GetId(const vertex_t &v) const
inline fid_t GetFragId(const vertex_t &u) const
inline size_t GetTotalNodesNum() const
template<typename T>
inline T GetData(const vertex_t &v, prop_id_t prop_id) const
inline bool HasChild(const vertex_t &v, label_id_t e_label) const
inline bool HasParent(const vertex_t &v, label_id_t e_label) const
inline int GetLocalOutDegree(const vertex_t &v, label_id_t e_label) const
inline int GetLocalInDegree(const vertex_t &v, label_id_t e_label) const
inline bool Gid2Vertex(const vid_t &gid, vertex_t &v) const
inline vid_t Vertex2Gid(const vertex_t &v) const
inline vid_t GetInnerVerticesNum(label_id_t label_id) const
inline vid_t GetOuterVerticesNum(label_id_t label_id) const
inline bool IsInnerVertex(const vertex_t &v) const
inline bool IsOuterVertex(const vertex_t &v) const
inline bool GetInnerVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
inline bool GetOuterVertex(label_id_t label, const oid_t &oid, vertex_t &v) const
inline oid_t GetInnerVertexId(const vertex_t &v) const
inline oid_t GetOuterVertexId(const vertex_t &v) const
inline oid_t Gid2Oid(const vid_t &gid) const
inline bool Oid2Gid(label_id_t label, const oid_t &oid, vid_t &gid) const
inline bool InnerVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
inline bool OuterVertexGid2Vertex(const vid_t &gid, vertex_t &v) const
inline vid_t GetOuterVertexGid(const vertex_t &v) const
inline vid_t GetInnerVertexGid(const vertex_t &v) const
inline adj_list_t GetIncomingAdjList(const vertex_t &v, label_id_t e_label) const
inline adj_list_t GetOutgoingAdjList(const vertex_t &v, label_id_t e_label) const
inline extra_adj_list_t GetExtraOutgoingAdjList(const vertex_t &v, label_id_t e_label) const
inline extra_adj_list_t GetExtraIncomingAdjList(const vertex_t &v, label_id_t e_label) const
inline grape::DestList IEDests(const vertex_t &v, label_id_t e_label) const
inline grape::DestList OEDests(const vertex_t &v, label_id_t e_label) const
inline grape::DestList IOEDests(const vertex_t &v, label_id_t e_label) const
inline std::shared_ptr<vertex_map_t> GetVertexMap()
inline void PrepareToRunApp(grape::MessageStrategy strategy, bool need_split_edges)
inline std::shared_ptr<extra_vertex_map_t> GetExtraVertexMap()
inline std::shared_ptr<AppendOnlyArrowTable> extra_edge_data_table(label_id_t i) const

Public Static Functions

static inline std::shared_ptr<vineyard::Object> Create()
template<typename OID_T, typename VID_T>
class gs::ArrowProjectedVertexMap : public vineyard::Registered<ArrowProjectedVertexMap<OID_T, VID_T>>

This class represents the mapping between oid and vid.

tparam OID_T

OID type

tparam VID_T

VID type

Public Functions

inline void Construct(const vineyard::ObjectMeta &meta)
inline bool GetOid(vid_t gid, oid_t &oid) const
inline bool GetGid(fid_t fid, oid_t oid, vid_t &gid) const
inline bool GetGid(oid_t oid, vid_t &gid) const
inline size_t GetTotalVerticesNum() const

Public Static Functions

static inline std::shared_ptr<vineyard::Object> Create()
static inline std::shared_ptr<ArrowProjectedVertexMap<OID_T, VID_T>> Project(std::shared_ptr<vineyard::ArrowVertexMap<OID_T, VID_T>> vm, label_id_t v_label)
template<typename OID_T = vineyard::property_graph_types::OID_TYPE, typename VID_T = vineyard::property_graph_types::VID_TYPE>
class gs::ArrowFragmentLoader

This loader can load a ArrowFragment from the data source including local file, oss, numpy, pandas and vineyard.

tparam OID_T

OID type

tparam VID_T

VID type

Public Functions

inline ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, const std::vector<std::string> &efiles, const std::vector<std::string> &vfiles, bool directed = true)
inline ArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, std::shared_ptr<detail::Graph> graph_info)
~ArrowFragmentLoader() = default
inline boost::leaf::result<std::vector<std::shared_ptr<arrow::Table>>> LoadVertexTables()
inline boost::leaf::result<std::vector<std::vector<std::shared_ptr<arrow::Table>>>> LoadEdgeTables()
inline boost::leaf::result<vineyard::ObjectID> AddLabelsToGraph(vineyard::ObjectID frag_id)
inline boost::leaf::result<vineyard::ObjectID> addVerticesAndEdges(vineyard::ObjectID frag_id)
inline boost::leaf::result<vineyard::ObjectID> addVertices(vineyard::ObjectID frag_id)
inline boost::leaf::result<vineyard::ObjectID> addEdges(vineyard::ObjectID frag_id)
inline boost::leaf::result<vineyard::ObjectID> LoadFragment()
inline boost::leaf::result<vineyard::ObjectID> AddLabelsToGraphAsFragmentGroup(vineyard::ObjectID frag_id)
inline boost::leaf::result<vineyard::ObjectID> LoadFragmentAsFragmentGroup()
inline boost::leaf::result<partitioner_t> initPartitioner()
template<typename OID_T, typename VID_T>
class gs::ArrowFragmentAppender

ArrowFragmentAppender is a utility to modify AppendOnlyArrowFragment in append fashion.

tparam OID_T

tparam VID_T

Public Functions

inline explicit ArrowFragmentAppender(grape::CommSpec &comm_spec, std::shared_ptr<AppendOnlyArrowFragment<OID_T, VID_T>> fragment)
inline bl::result<int64_t> ExtendFragment(std::vector<std::vector<std::string>> &vertex_messages, std::vector<std::vector<std::string>> &edge_messages, bool header_row, char delimiter, bool directed)

Only should be invoked on Coordinator process

Parameters
  • vertex_messages

  • edge_messages

  • directed

template<typename OID_T, typename VID_T>
class gs::AppendOnlyArrowFragmentBuilder

Subclassed by gs::BasicAppendOnlyArrowFragmentBuilder< OID_T, VID_T >

Public Functions

inline explicit AppendOnlyArrowFragmentBuilder(vineyard::Client &client)
inline void set_fid(fid_t fid)
inline void set_fnum(fid_t fnum)
inline void set_directed(bool directed)
inline void set_label_num(label_id_t vertex_label_num, label_id_t edge_label_num)
inline void set_ivnums(const vineyard::Array<vid_t> &ivnums)
inline void set_ovnums(const vineyard::Array<vid_t> &ovnums)
inline void set_tvnums(const vineyard::Array<vid_t> &tvnums)
inline void set_vertex_table(label_id_t label, std::shared_ptr<vineyard::Table> table)
inline void set_ovgid_list(label_id_t label, std::shared_ptr<vineyard::NumericArray<vid_t>> ovgid_list)
inline void set_ovg2l_map(label_id_t label, std::shared_ptr<vineyard::Hashmap<vid_t, vid_t>> ovg2l_map)
inline void set_edge_table(label_id_t label, std::shared_ptr<vineyard::Table> table)
inline void set_in_edge_list(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::FixedSizeBinaryArray> in_edge_list)
inline void set_out_edge_list(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::FixedSizeBinaryArray> out_edge_list)
inline void set_in_edge_offsets(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::NumericArray<int64_t>> in_edge_offsets)
inline void set_out_edge_offsets(label_id_t v_label, label_id_t e_label, std::shared_ptr<vineyard::NumericArray<int64_t>> out_edge_offsets)
inline void set_vertex_map(std::shared_ptr<vertex_map_t> vm_ptr)
inline std::shared_ptr<vineyard::Object> _Seal(vineyard::Client &client) override
template<typename OID_T = vineyard::property_graph_types::OID_TYPE, typename VID_T = vineyard::property_graph_types::VID_TYPE>
class gs::AppendOnlyArrowFragmentLoader

AppendOnlyArrowFragmentLoader loads AppendOnlyArrowFragment from local files.

tparam OID_T

tparam VID_T

Public Functions

inline AppendOnlyArrowFragmentLoader(vineyard::Client &client, const grape::CommSpec &comm_spec, label_id_t vertex_label_num, label_id_t edge_label_num, std::string efile, std::string vfile, bool directed = true)
~AppendOnlyArrowFragmentLoader() = default
inline bl::result<vineyard::ObjectID> LoadFragment()

Analytical Apps

template<typename FRAG_T, typename CONTEXT_T>
class gs::AppBase

AppBase is the base class for applications.

It contains an grape::DefaultMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

tparam FRAG_T

Fragment class

tparam CONTEXT_T

App context class

Public Types

using message_manager_t = grape::DefaultMessageManager

Public Functions

AppBase() = default
virtual ~AppBase() = default
virtual void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

Public Static Attributes

static constexpr bool need_split_edges = false
static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kOnlyOut
template<typename FRAG_T, typename CONTEXT_T>
class gs::PropertyAppBase

PropertyAppBase is the base class for applications on property graphs.

It contains an PropertyMessageManager to process messages during execution of application. Apps derived from this class need to process messages explicitly.

tparam FRAG_T

tparam CONTEXT_T

Public Types

using message_manager_t = PropertyMessageManager

Public Functions

PropertyAppBase() = default
virtual ~PropertyAppBase() = default
virtual void PEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context, message_manager_t &messages) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

Public Static Attributes

static constexpr bool need_split_edges = false
static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename FRAG_T, typename CONTEXT_T>
class gs::PropertyAutoAppBase

PropertyAutoAppBase is the base class for applications on property graph, with an PropertyAutoMessageManager to process messages during execution of application. Apps derived from this class do not need to process messages explicitly.

tparam FRAG_T

tparam CONTEXT_T

Public Functions

PropertyAutoAppBase() = default
virtual ~PropertyAutoAppBase() = default
virtual void PEval(const FRAG_T &graph, CONTEXT_T &context) = 0

Partial evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The PEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

virtual void IncEval(const FRAG_T &graph, CONTEXT_T &context) = 0

Incremental evaluation to implement.

Note

: This pure virtual function works as an interface, instructing users to implement in the specific app. The IncEval in the inherited apps would be invoked directly, not via virtual functions.

Parameters
  • graph

  • context

  • messages

Public Static Attributes

static constexpr bool need_split_edges = false
static constexpr grape::MessageStrategy message_strategy = grape::MessageStrategy::kSyncOnOuterVertex
static constexpr grape::LoadStrategy load_strategy = grape::LoadStrategy::kBothOutIn
template<typename APP_T>
class gs::DefaultWorker

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = grape::DefaultMessageManager

Public Functions

inline DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
template<typename APP_T>
class gs::ParallelPropertyWorker

A parallel worker for labeled fragment.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = ParallelPropertyMessageManager

Public Functions

inline ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
virtual ~ParallelPropertyWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
template<typename APP_T>
class gs::PropertyAutoWorker

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>

Public Functions

inline PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~PropertyAutoWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
class gs::AppEntry : public gs::GSObject

AppEntry is a class manages an application.

AppEntry holds the a group of function pointers to manipulate the AppFrame, such as gs::CreateWorker, DeleteWorker and Query. The method Init must be called to load the library before use.

Public Functions

inline AppEntry(std::string id, std::string lib_path)
inline bl::result<void> Init()
inline bl::result<std::shared_ptr<void>> CreateWorker(const std::shared_ptr<void> &fragment, const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &spec)
inline bl::result<std::shared_ptr<IContextWrapper>> Query(void *worker_handler, const rpc::QueryArgs &query_args, const std::string &context_key, std::shared_ptr<IFragmentWrapper> &frag_wrapper)
template<class FunctionType>
struct ArgsNum
template<typename T>
struct ArgsUnpacker

ArgsUnpacker is an util to unpack the Any type of proto to C++ datatype.

tparam T

The destination type

template<std::size_t index, class FunctionType>
struct ArgTypeAt

Analytical Workers

template<typename APP_T>
class gs::DefaultWorker

DefaultWorker manages the computation cycle. DefaultWorker is a kind of serial worker for apps derived from AppBase.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = grape::DefaultMessageManager

Public Functions

inline DefaultWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
template<typename APP_T>
class gs::DefaultPropertyWorker

A Worker manages the computation cycle. DefaultPropertyWorker is a kind of worker for apps derived from AppBase.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyMessageManager

Public Functions

inline DefaultPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~DefaultPropertyWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
template<typename APP_T>
class gs::ParallelPropertyWorker

A parallel worker for labeled fragment.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = ParallelPropertyMessageManager

Public Functions

inline ParallelPropertyWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
virtual ~ParallelPropertyWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)
template<typename APP_T>
class gs::PropertyAutoWorker

A Worker manages the computation cycle. PropertyAutoWorker is a kind of worker for apps derived from PropertyAutoAppBase.

tparam APP_T

Public Types

using fragment_t = typename APP_T::fragment_t
using context_t = typename APP_T::context_t
using message_manager_t = PropertyAutoMessageManager<typename APP_T::fragment_t>

Public Functions

inline PropertyAutoWorker(std::shared_ptr<APP_T> app, std::shared_ptr<fragment_t> graph)
~PropertyAutoWorker() = default
inline void Init(const grape::CommSpec &comm_spec, const grape::ParallelEngineSpec &pe_spec = grape::DefaultParallelEngineSpec())
inline void Finalize()
template<class ...Args>
inline void Query(Args&&... args)
inline std::shared_ptr<context_t> GetContext()
inline void Output(std::ostream &os)

App’s Context

template<typename CTX_T, typename = void>
struct CtxWrapperBuilder

This is a utility to build kinds of ContextWrapper instances.

tparam CTX_T

template<typename FRAG_T>
class gs::VertexPropertyContext : public ContextBase

VertexPropertyContext can hold any number of columns. The context is designed for labeled fragment - ArrowFragment. Compared with LabeledVertexDataContext, the data type and column count can be determined at runtime.

tparam FRAG_T

The fragment class (non-labeled fragment only)

Public Types

using fragment_t = FRAG_T
using vertex_t = typename fragment_t::vertex_t
using prop_id_t = vineyard::property_graph_types::PROP_ID_TYPE
using oid_t = typename fragment_t::oid_t

Public Functions

inline explicit VertexPropertyContext(const fragment_t &fragment)
inline const fragment_t &fragment()
inline int64_t add_column(const std::string &name, ContextDataType type)
inline std::shared_ptr<IColumn> get_column(int64_t index)
inline std::shared_ptr<IColumn> get_column(const std::string &name)
template<typename DATA_T>
inline std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(int64_t index)
template<typename DATA_T>
inline std::shared_ptr<Column<fragment_t, DATA_T>> get_typed_column(const std::string &name)
inline std::vector<std::shared_ptr<IColumn>> &vertex_properties()
inline const std::map<std::string, std::shared_ptr<IColumn>> &properties_map()
template<typename FRAG_T, typename DATA_T>
class gs::VertexDataContextWrapper : public gs::IVertexDataContextWrapper

This is the wrapper class for VertexDataContext. A series of methods are provided to transform the data hold by the context.

tparam FRAG_T

The fragment class (Non-labeled fragment only)

tparam DATA_T

The Data type hold by context

Public Functions

inline explicit VertexDataContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)
inline virtual std::string context_type() override
inline virtual std::shared_ptr<IFragmentWrapper> fragment_wrapper() override
inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, const Selector &selector, const std::pair<std::string, std::string> &range) override
inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override
inline virtual bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, const Selector &selector, const std::pair<std::string, std::string> &range) override
inline virtual bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client, const std::vector<std::pair<std::string, Selector>> &selectors, const std::pair<std::string, std::string> &range) override
inline virtual bl::result<std::vector<std::pair<std::string, std::shared_ptr<arrow::Array>>>> ToArrowArrays(const grape::CommSpec &comm_spec, const std::vector<std::pair<std::string, Selector>> &selectors) override
template<typename FRAG_T, typename DATA_T>
class gs::TensorContext : public ContextBase

TensorContext is designed for holding a bunch of computation results. The TensorContext should be used if the number of elements are not related to the number of the vertex.

tparam FRAG_T

tparam DATA_T

Public Types

using data_t = DATA_T

Public Functions

inline explicit TensorContext(const fragment_t &fragment)
inline const fragment_t &fragment()
inline void assign(const std::vector<data_t> &data, const std::vector<size_t> &shape)
inline void assign(const data_t &data)
inline void set_shape(std::vector<std::size_t> shape)
inline std::vector<size_t> shape() const
inline tensor_t &tensor()
template<typename FRAG_T, typename DATA_T, typename = void>
class gs::TensorContextWrapper : public gs::ITensorContextWrapper

TensorContextWrapper is the wrapper class for TensorContext.

tparam FRAG_T

tparam DATA_T

Public Functions

inline explicit TensorContextWrapper(const std::string &id, std::shared_ptr<IFragmentWrapper> frag_wrapper, std::shared_ptr<context_t> ctx)
inline virtual std::string context_type() override
inline virtual std::shared_ptr<IFragmentWrapper> fragment_wrapper() override
inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToNdArray(const grape::CommSpec &comm_spec, uint32_t axis) override
inline virtual bl::result<std::unique_ptr<grape::InArchive>> ToDataframe(const grape::CommSpec &comm_spec) override
inline virtual bl::result<vineyard::ObjectID> ToVineyardTensor(const grape::CommSpec &comm_spec, vineyard::Client &client, uint32_t axis) override
inline virtual bl::result<vineyard::ObjectID> ToVineyardDataframe(const grape::CommSpec &comm_spec, vineyard::Client &client) override