如何在Neo4j中创建数十亿个节点？

我想用大量节点测试Neo4j性能.我正在考虑创建数十亿个节点,然后想要查看获取符合某些条件的节点所需的时间.就像10亿个标记为具有SSN属性的人一样

match (p:Person) where p.SSN=4255556656425 return p;

但是,如何创建10亿个节点,有没有办法生成10亿个节点？

那么你要测量的是lucene指数的表现.所以不是图形数据库操作.

有很多选择:

Neo4j的进口

Neo4j 2.2.0-M03附带neo4j-import,这个工具可以快速,可扩展地将10亿个节点csv导入Neo4j.

并行批处理导入程序API

这在Neo4j 2.2中是非常新的

我使用新的ParallelBatchImporter在5分13秒(53G db)中创建了一个仅有节点的图,其中包含1.000.000.000个节点.这使得它约为3.2M节点/秒.

代码在这里:https://gist.github.com/jexp/0ff850ab2ce41c9ca5e6

批量插入

您可以使用Neo4j Batch-Inserter-API创建该数据,而无需先创建CSV.

请参阅此示例,您必须采用该示例才能读取CSV但直接从for循环生成数据:http://jexp.de/blog/2014/10/flexible-neo4j-batch-import-with-groovy/

暗号

如果你想使用Cypher,我建议你在下面运行JAVA_OPTS="-Xmx4G -Xms4G" bin/neo4j-shell -path billion.db:

以下是我在macbook上使用的10M和100M的代码和时序:

创建一个包含1M行的csv文件

ruby -e 'File.open("million.csv","w") 
   { |f| (1..1000000).each{|i| f.write(i.to_s + "\n") }  }' 

在MacBook Pro上执行的实验Cypher执行是单线程估计大小(15 + 42)字节*节点计数

// on my laptop
// 10M nodes, 1 property, 1 label each in 98228 ms (98s) taking 580 MB on disk

using periodic commit 10000
load csv from "file:million.csv" as row
//with row limit 5000
foreach (x in range(0,9) | create (:Person {id:toInt(row[0])*10+x}));

// on my laptop
// 100M nodes, 1 property, 1 label each in 1684411 ms (28 mins) taking 6 GB on disk

using periodic commit 1000
load csv from "file:million.csv" as row
foreach (x in range(0,99) | create (:Person {id:toInt(row[0])*100+x}));

// on my linux server
// 1B nodes, 1 property, 1 label each in 10588883 ms (176 min) taking 63 GB on disk

using periodic commit 1000
load csv from "file:million.csv" as row
foreach (x in range(0,999) | create (:Person {id:toInt(row[0])*100+x}));

创建索引

create index on :Person(id);
schema await

// took about 40 mins and increased the database size to 85 GB

然后我就可以跑了

match (:Person {id:8005300}) return count(*);
+----------+
| count(*) |
+----------+
| 1        |
+----------+
1 row
2 ms

另一个简单的答案是好的.如果你想要更多涉及的东西,Michael Hunger就此发布了一篇很好的博客文章.他推荐的东西基本上非常相似,但你也可以循环一些样本数据,并使用随机数建立联系.

以下是他如何创建100,000个用户和产品并将其链接起来,根据您的需要进行自定义:

WITH ["Andres","Wes","Rik","Mark","Peter","Kenny","Michael","Stefan","Max","Chris"] AS names
FOREACH (r IN range(0,100000) | CREATE (:User {id:r, name:names[r % size(names)]+" "+r}));

with ["Mac","iPhone","Das Keyboard","Kymera Wand","HyperJuice Battery",
"Peachy Printer","HexaAirBot",
"AR-Drone","Sonic Screwdriver",
"Zentable","PowerUp"] as names
    foreach (r in range(0,50) | create (:Product {id:r, name:names[r % size(names)]+" "+r}));

让我们不要忘记甜蜜的随机链接:

match (u:User),(p:Product)
where rand() <0.1
with u,p
limit 50000
merge (u)-[:OWN]->(p);

发疯.