Graph Database Comparisons

[robobenklein]

Requirements:

• Must have native graph endpoints (to allow for serverside traversals, etc)
• Must be open source (free for use by both education and for-profit use cases)
• Must support many hundreds of TB of data (meaning many, many billions of nodes and edges, and must not grow too slow)

This thread will be a sort of list and discussion for database options, each top-level response should be one database option, with replies to discuss that option.

[robobenklein]

Neo4j

Failed at ~250GB of data stored, memory requirements became absurd, response times became unusable after a few hundred gigs of data and DB inserts became unreliable.

neo4j/neo4j#12686

Might want to go back and re-evaluate with jsonl file output data insertions.

[robobenklein]

ArangoDB

Currently performing well, unfortunately does not have a friendly query language for end users like cypher.

We have yet to hit any data limits or slowdowns after passing ~1TB of data stored.

[cadebrown]

I've been playing around with parsing using Lark (https://github.com/lark-parser/lark). It seems fairly straightforward to generate ArangoDB traversals from a higher-level cypher-like query language using either:

• a transformer (https://lark-parser.readthedocs.io/en/latest/visitors.html)
• parse tree conversion routine

Just playing around it seems simple to hack around on, fairly Pythonic, and provides error diagnostics easily

[robobenklein]

AgensGraph

Built on top of postgres? Supports OpenCypher, needs evaluation.

https://github.com/bitnine-oss/agensgraph

[robobenklein]

BlazeGraph

50 billion edge limit is too small, probably not going to attempt to evaluate.

https://github.com/blazegraph/database

[robobenklein]

Dgraph

Needs research

https://github.com/dgraph-io/dgraph

[robobenklein]

JanusGraph

Needs research

https://janusgraph.org/

[robobenklein]

Nebula

Advertises Cypher-compatible lang, needs research

https://github.com/vesoft-inc/nebula

[robobenklein]

OrientDB

Needs research

https://github.com/orientechnologies/orientdb

[cadebrown]

tutorial for graphs: https://orientdb.com/docs/last/gettingstarted/Tutorial-Working-with-graphs.html

looks like a SQL-like language (different than cypher), but seems like it would be easy to generate.

also, it looks like we shouldn't have too much of an issue with limits: https://orientdb.com/docs/2.2.x/Limits.html

[cadebrown]

also, they claim to have a full-text search. this could be extremely useful for a fuzzy

it requires a bit of code, but looks very hackable: https://stackoverflow.com/questions/29853405/lucene-acting-up-on-orientdb-when-confronted-with-fuzzy-queries

-1 for Java but +3 for the features