Sparkpipe+PHP的wordcount实现

php的工作就是把每个词搞成一个RDD

$in = fopen('php://stdin','r');
while(!feof($in))
{
  $temp = explode(" ",fgets($in));
  for ($i=0;$i$temp);$i++){
     printf("%s\n",$temp[$i]);  
  } 
}
?>

package test

import org.apache.spark.SparkConf
import org.apache.spark.SparkContextobject PipeTest {

  def main(args: Array[String]) {
    val sparkCOnf= new SparkConf().setAppName("pipe Test")
    val sc = new SparkContext(sparkConf)
    val a = sc.textFile("/home/gt/wordcount.txt", 3)
    val result = a.pipe("php /home/gt/spark/bin/test.php").map(x => (x, 1)).reduceByKey(_ + _)
    result.foreach { x => println("!!!!! " + x) }
    sc.stop()
  }

}

').addClass('pre-numbering').hide(); $(this).addClass('has-numbering').parent().append($numbering); for (i = 1; i <= lines; i++) { $numbering.append($('

').text(i)); }; $numbering.fadeIn(1700); }); });

以上就介绍了Spark pipe + PHP 的 wordcount 实现，包括了方面的内容，希望对PHP教程有兴趣的朋友有所帮助。