spark的任务已经执行完成:
scala> val line = sc.textFile("hdfs://vm122:9000/dblp.rdf")
line: org.apache.spark.rdd.RDD[String] = hdfs://vm122:9000/dblp.rdf MapPartitionsRDD[1] at textFile at :24scala> val n = line.map { x ⇒ 1 }.count()
n: Long = 150787524
然而监控界面一直显示RUNNING:
原因:
spark-shell在启动期间会一直RUNNING
进入Application Detail UI
可以看到不同的job执行状态