python英文关键词提取_如何提取文章的关键词（Python版）

项目需求&＃xff1a;

我们采集来的文章没有关键词&＃xff0c;在发布的时候无法设定标签&＃xff0c;我们通过代码自动提取出文章的关键词&＃xff0c;达到对数据加工的目的。

测试环境&＃xff1a;

Anaconda Python3.5

Win7 ultmate(英文版&＃xff09;

Excel 2013英文版本

用一百篇文章来测试。

一、使用snownlp&＃xff08;https://github.com/isnowfy/snownlp&＃xff09;

代码如下&＃xff1a;

import xlrd

from snownlp import SnowNLP

data &＃61; xlrd.open_workbook("article.xlsx")

table &＃61; data.sheets()[0]

nrows &＃61; table.nrows #行数

ncols &＃61; table.ncols #列数

for i in range(0,nrows):

rowValues&＃61; table.row_values(i) #某一行数据

for item in rowValues:

# print(item)

s &＃61; SnowNLP(item)

print(s.keywords(3))

执行结果&＃xff1a;

[&＃39;不&＃39;, &＃39;棋子&＃39;, &＃39;梦&＃39;]

[&＃39;姑娘&＃39;, &＃39;说&＃39;, &＃39;达&＃39;]

[&＃39;很&＃39;, &＃39;不&＃39;, &＃39;会&＃39;]

[&＃39;很&＃39;, &＃39;不&＃39;, &＃39;会&＃39;]

[&＃39;很&＃39;, &＃39;不&＃39;, &＃39;会&＃39;]

[&＃39;生日&＃39;, &＃39;快乐&＃39;, &＃39;走&＃39;]

[&＃39;不&＃39;, &＃39;发现&＃39;, &＃39;一个&＃39;]

[&＃39;爱&＃39;, &＃39;不&＃39;, &＃39;笨&＃39;]

[&＃39;事&＃39;, &＃39;每次&＃39;, &＃39;都&＃39;]

[&＃39;...&＃39;, &＃39;天气&＃39;, &＃39;&＃xff09;&＃xff1a;&＃39;]

[&＃39;不&＃39;, &＃39;最&＃39;, &＃39;人&＃39;]

[&＃39;会&＃39;, &＃39;特&＃39;, &＃39;还&＃39;]

[&＃39;吵&＃39;, &＃39;不&＃39;, &＃39;背&＃39;]

[&＃39;吵&＃39;, &＃39;不&＃39;, &＃39;背&＃39;]

[&＃39;不&＃39;, &＃39;&＃xff0e;&＃39;, &＃39;进来&＃39;]

[&＃39;男孩&＃39;, &＃39;1&＃39;, &＃39;信&＃39;]

[&＃39;不&＃39;, &＃39;上&＃39;, &＃39;生活&＃39;]

[&＃39;英语&＃39;, &＃39;说&＃39;, &＃39;坐&＃39;]

[&＃39;不&＃39;, &＃39;去&＃39;, &＃39;没&＃39;]

[&＃39;不&＃39;, &＃39;朋友&＃39;, &＃39;已&＃39;]

[&＃39;幸福&＃39;, &＃39;掉&＃39;, &＃39;最&＃39;]

[&＃39;不&＃39;, &＃39;想&＃39;, &＃39;陪&＃39;]

[&＃39;没&＃39;, &＃39;说&＃39;, &＃39;真&＃39;]

[&＃39;哥哥&＃39;, &＃39;很&＃39;, &＃39;妈妈&＃39;]

[&＃39;会&＃39;, &＃39;不&＃39;, &＃39;很&＃39;]

[&＃39;不&＃39;, &＃39;很&＃39;, &＃39;都&＃39;]

[&＃39;不&＃39;, &＃39;好&＃39;, &＃39;都&＃39;]

[&＃39;很&＃39;, &＃39;不&＃39;, &＃39;却&＃39;]

[&＃39;不&＃39;, &＃39;确实&＃39;, &＃39;去&＃39;]

[&＃39;不&＃39;, &＃39;都&＃39;, &＃39;很&＃39;]

[&＃39;不&＃39;, &＃39;说&＃39;, &＃39;一个&＃39;]

[&＃39;家庭&＃39;, &＃39;不&＃39;, &＃39;生活&＃39;]

[&＃39;不&＃39;, &＃39;会&＃39;, &＃39;想&＃39;]

[&＃39;哭&＃39;, &＃39;永远&＃39;, &＃39;好&＃39;]

[&＃39;不&＃39;, &＃39;会&＃39;, &＃39;一个&＃39;]

[&＃39;不&＃39;, &＃39;人&＃39;, &＃39;爱&＃39;]

[&＃39;现实&＃39;, &＃39;相处&＃39;, &＃39;游戏&＃39;]

[&＃39;告诉&＃39;, &＃39;不&＃39;, &＃39;都&＃39;]

[&＃39;很&＃39;, &＃39;一个&＃39;, &＃39;说&＃39;]

[&＃39;告诉&＃39;, &＃39;小&＃39;, &＃39;幸福&＃39;]

[&＃39;很&＃39;, &＃39;公&＃39;, &＃39;带&＃39;]

[&＃39;不&＃39;, &＃39;帮&＃39;, &＃39;哥哥&＃39;]

[&＃39;不&＃39;, &＃39;难&＃39;, &＃39;爱&＃39;]

[&＃39;不&＃39;, &＃39;分手&＃39;, &＃39;后&＃39;]

[&＃39;不&＃39;, &＃39;都&＃39;, &＃39;没&＃39;]

[&＃39;玩&＃39;, &＃39;不&＃39;, &＃39;记得&＃39;]

[&＃39;去&＃39;, &＃39;事情&＃39;, &＃39;更&＃39;]

[&＃39;会&＃39;, &＃39;不&＃39;, &＃39;很&＃39;]

[&＃39;很&＃39;, &＃39;竟然&＃39;, &＃39;不&＃39;]

[&＃39;说&＃39;, &＃39;上&＃39;, &＃39;信息&＃39;]

[&＃39;不&＃39;, &＃39;家里&＃39;, &＃39;说&＃39;]

[&＃39;不&＃39;, &＃39;都&＃39;, &＃39;再&＃39;]

[&＃39;不&＃39;, &＃39;工作&＃39;, &＃39;感觉&＃39;]

[&＃39;不&＃39;, &＃39;工作&＃39;, &＃39;感觉&＃39;]

[&＃39;会&＃39;, &＃39;次&＃39;, &＃39;好&＃39;]

[&＃39;公主&＃39;, &＃39;王子&＃39;, &＃39;一个&＃39;]

[&＃39;感谢&＃39;, &＃39;人生&＃39;, &＃39;年&＃39;]

[&＃39;感谢&＃39;, &＃39;人生&＃39;, &＃39;年&＃39;]

[&＃39;好&＃39;, &＃39;……&＃39;, &＃39;嗎&＃39;]

[&＃39;一个&＃39;, &＃39;好&＃39;, &＃39;很&＃39;]

[&＃39;人&＃39;, &＃39;死&＃39;, &＃39;女人&＃39;]

[&＃39;人&＃39;, &＃39;不&＃39;, &＃39;朋友&＃39;]

[&＃39;想&＃39;, &＃39;都&＃39;, &＃39;不&＃39;]

[&＃39;不&＃39;, &＃39;想&＃39;, &＃39;分开&＃39;]

[&＃39;真的&＃39;, &＃39;爱&＃39;, &＃39;很&＃39;]

[&＃39;旳&＃39;, &＃39;尓&＃39;, &＃39;不&＃39;]

[&＃39;说&＃39;, &＃39;座&＃39;, &＃39;不&＃39;]

[&＃39;说&＃39;, &＃39;不&＃39;, &＃39;朋友&＃39;]

[&＃39;最&＃39;, &＃39;年&＃39;, &＃39;疼爱&＃39;]

[&＃39;记得&＃39;, &＃39;cc&＃39;, &＃39;别怪&＃39;]

[&＃39;不&＃39;, &＃39;很&＃39;, &＃39;想&＃39;]

[&＃39;不&＃39;, &＃39;妈&＃39;, &＃39;回&＃39;]

[&＃39;不&＃39;, &＃39;人&＃39;, &＃39;都&＃39;]

[&＃39;不&＃39;, &＃39;寒&＃39;, &＃39;会&＃39;]

[&＃39;不&＃39;, &＃39;想&＃39;, &＃39;出轨&＃39;]

[&＃39;很&＃39;, &＃39;A&＃39;, &＃39;好&＃39;]

[&＃39;还&＃39;, &＃39;都&＃39;, &＃39;理由&＃39;]

[&＃39;想&＃39;, &＃39;偷偷&＃39;, &＃39;还&＃39;]

[&＃39;很&＃39;, &＃39;哥哥&＃39;, &＃39;大&＃39;]

[&＃39;不&＃39;, &＃39;人&＃39;, &＃39;公&＃39;]

[&＃39;不&＃39;, &＃39;子&＃39;, &＃39;会&＃39;]

[&＃39;朋友&＃39;, &＃39;很&＃39;, &＃39;去&＃39;]

[&＃39;年&＃39;, &＃39;不&＃39;, &＃39;责怪&＃39;]

[&＃39;不&＃39;, &＃39;聊&＃39;, &＃39;还&＃39;]

[&＃39;学校&＃39;, &＃39;正好&＃39;, &＃39;都&＃39;]

[&＃39;会&＃39;, &＃39;不&＃39;, &＃39;好&＃39;]

[&＃39;说&＃39;, &＃39;很&＃39;, &＃39;不&＃39;]

[&＃39;不&＃39;, &＃39;会&＃39;, &＃39;没&＃39;]

[&＃39;人&＃39;, &＃39;不&＃39;, &＃39;生活&＃39;]

[&＃39;不&＃39;, &＃39;我怕&＃39;, &＃39;依然&＃39;]

[&＃39;不&＃39;, &＃39;我怕&＃39;, &＃39;依然&＃39;]

[&＃39;玩&＃39;, &＃39;不&＃39;, &＃39;55555555555555555

[&＃39;姑娘&＃39;, &＃39;不&＃39;, &＃39;却&＃39;]

[&＃39;种&＃39;, &＃39;一个&＃39;, &＃39;想法&＃39;]

[&＃39;朋友&＃39;, &＃39;Y&＃39;, &＃39;..&＃39;]

[&＃39;会&＃39;, &＃39;永远&＃39;, &＃39;不&＃39;]

[&＃39;说&＃39;, &＃39;真的&＃39;, &＃39;很&＃39;]

[&＃39;不&＃39;, &＃39;还&＃39;, &＃39;真&＃39;]

[&＃39;吃&＃39;, &＃39;去&＃39;, &＃39;人&＃39;]

[&＃39;不&＃39;, &＃39;谈&＃39;, &＃39;发现&＃39;]

可以看到&＃xff0c;这个出现了很多的单个词&＃xff0c;效果并不是很好。

二、使用jieba

利用pip安装jieba&＃xff0c;可能网速的问题&＃xff0c;没有安装上&＃xff0c;后来使用本地安装的方法&＃xff0c;终于装上了。

代码如下&＃xff1a;

# -*- coding: utf-8 -*-

import xlrd

from jieba import analyse

data &＃61; xlrd.open_workbook("article.xlsx")

table &＃61; data.sheets()[0]

nrows &＃61; table.nrows #行数

ncols &＃61; table.ncols #列数

for i in range(0,nrows):

rowValues&＃61; table.row_values(i) #某一行数据

for item in rowValues:

# print(item)

tfidf &＃61; analyse.extract_tags

keywords &＃61; tfidf(item)

#for keyword in keywords:

print (keywords[:3])

执行结果&＃xff1a;

[&＃39;亲情&＃39;, &＃39;棋子&＃39;, &＃39;快乐&＃39;]

[&＃39;爱过&＃39;, &＃39;姑娘&＃39;, &＃39;他会&＃39;]

[&＃39;但会&＃39;, &＃39;喜欢&＃39;, &＃39;我们&＃39;]

[&＃39;但会&＃39;, &＃39;喜欢&＃39;, &＃39;我们&＃39;]

[&＃39;但会&＃39;, &＃39;喜欢&＃39;, &＃39;我们&＃39;]

[&＃39;生日快乐&＃39;, &＃39;生日&＃39;, &＃39;以为&＃39;]

[&＃39;其它&＃39;, &＃39;她们&＃39;, &＃39;我动&＃39;]

[&＃39;mdash&＃39;, &＃39;多么&＃39;, &＃39;牵起&＃39;]

[&＃39;每次&＃39;, &＃39;女朋友&＃39;, &＃39;似乎&＃39;]

[&＃39;...&＃39;, &＃39;那个&＃39;, &＃39;天气&＃39;]

[&＃39;最深&＃39;, &＃39;依赖&＃39;, &＃39;看清&＃39;]

[&＃39;走进&＃39;, &＃39;一起&＃39;, &＃39;还会&＃39;]

[&＃39;爱情&＃39;, &＃39;因为&＃39;, &＃39;争吵&＃39;]

[&＃39;爱情&＃39;, &＃39;因为&＃39;, &＃39;争吵&＃39;]

[&＃39;围城&＃39;, &＃39;那么&＃39;, &＃39;冲满&＃39;]

[&＃39;男孩&＃39;, &＃39;异地&＃39;, &＃39;......&＃39;]

[&＃39;可是&＃39;, &＃39;可以&＃39;, &＃39;因为&＃39;]

[&＃39;英语&＃39;, &＃39;作业&＃39;, &＃39;初二&＃39;]

[&＃39;因为&＃39;, &＃39;可是&＃39;, &＃39;扣扣&＃39;]

[&＃39;朋友&＃39;, &＃39;珍惜&＃39;, &＃39;放手&＃39;]

[&＃39;删掉&＃39;, &＃39;幸福&＃39;, &＃39;想要&＃39;]

[&＃39;一直&＃39;, &＃39;这样&＃39;, &＃39;不去&＃39;]

[&＃39;喜欢&＃39;, &＃39;没说&＃39;, &＃39;嘻嘻&＃39;]

[&＃39;哥哥&＃39;, &＃39;妈妈&＃39;, &＃39;越来越&＃39;]

[&＃39;可以&＃39;, &＃39;默默&＃39;, &＃39;真的&＃39;]

[&＃39;男朋友&＃39;, &＃39;上班&＃39;, &＃39;光棍节&＃39;]

[&＃39;有没有&＃39;, &＃39;认真听讲&＃39;, &＃39;__&＃39;]

[&＃39;狠狠&＃39;, &＃39;抱住&＃39;, &＃39;伤害&＃39;]

[&＃39;自己&＃39;, &＃39;爸妈&＃39;, &＃39;确实&＃39;]

[&＃39;我们&＃39;, &＃39;这样&＃39;, &＃39;可以&＃39;]

[&＃39;亲爱&＃39;, &＃39;放弃&＃39;, &＃39;早就&＃39;]

[&＃39;真的&＃39;, &＃39;从小&＃39;, &＃39;路上&＃39;]

[&＃39;等待&＃39;, &＃39;见到&＃39;, &＃39;然不&＃39;]

[&＃39;你们&＃39;, &＃39;永远&＃39;, &＃39;我视&＃39;]

[&＃39;内个&＃39;, &＃39;表达&＃39;, &＃39;六班&＃39;]

[&＃39;喜欢&＃39;, &＃39;爱情&＃39;, &＃39;一帆风顺&＃39;]

[&＃39;杨棋文&＃39;, &＃39;放不下&＃39;, &＃39;现实&＃39;]

[&＃39;媳妇&＃39;, &＃39;起码&＃39;, &＃39;告诉&＃39;]

[&＃39;胖胖的&＃39;, &＃39;害怕&＃39;, &＃39;放学&＃39;]

[&＃39;为什么&＃39;, &＃39;灰灰&＃39;, &＃39;卡是&＃39;]

[&＃39;结婚&＃39;, &＃39;母亲&＃39;, &＃39;老公&＃39;]

[&＃39;为什么&＃39;, &＃39;晴子&＃39;, &＃39;暗恋&＃39;]

[&＃39;那么&＃39;, &＃39;想见&＃39;, &＃39;放手&＃39;]

[&＃39;男生&＃39;, &＃39;分手&＃39;, &＃39;知道&＃39;]

[&＃39;无缘&＃39;, &＃39;发现&＃39;, &＃39;或许&＃39;]

[&＃39;一起&＃39;, &＃39;记得&＃39;, &＃39;喜欢&＃39;]

[&＃39;学会&＃39;, &＃39;真的&＃39;, &＃39;事情&＃39;]

[&＃39;不会&＃39;, &＃39;但是&＃39;, &＃39;确说&＃39;]

[&＃39;竟然&＃39;, &＃39;出轨&＃39;, &＃39;真的&＃39;]

[&＃39;信息&＃39;, &＃39;宝宝&＃39;, &＃39;手机&＃39;]

[&＃39;打工&＃39;, &＃39;你家&＃39;, &＃39;不了&＃39;]

[&＃39;心死&＃39;, &＃39;野合&＃39;, &＃39;健健康康&＃39;]

[&＃39;自己&＃39;, &＃39;感觉&＃39;, &＃39;工作&＃39;]

[&＃39;自己&＃39;, &＃39;感觉&＃39;, &＃39;工作&＃39;]

[&＃39;真的&＃39;, &＃39;打扰&＃39;, &＃39;见面&＃39;]

[&＃39;公主&＃39;, &＃39;王子&＃39;, &＃39;分手&＃39;]

[&＃39;感谢&＃39;, &＃39;2011&＃39;, &＃39;感言&＃39;]

[&＃39;感谢&＃39;, &＃39;2011&＃39;, &＃39;感言&＃39;]

[&＃39;覺得&＃39;, &＃39;身邊&＃39;, &＃39;離開&＃39;]

[&＃39;偷偷地&＃39;, &＃39;三班&＃39;, &＃39;表白&＃39;]

[&＃39;出轨&＃39;, &＃39;昨晚&＃39;, &＃39;名字&＃39;]

[&＃39;一個&＃39;, &＃39;還是&＃39;, &＃39;結果&＃39;]

[&＃39;很想&＃39;, &＃39;自己&＃39;, &＃39;想要&＃39;]

[&＃39;为什么&＃39;, &＃39;不是&＃39;, &＃39;分开&＃39;]

[&＃39;真的&＃39;, &＃39;徐靖&＃39;, &＃39;我爱你&＃39;]

[&＃39;认识&＃39;, &＃39;因为&＃39;, &＃39;宝贝&＃39;]

[&＃39;我爱你&＃39;, &＃39;为何&＃39;, &＃39;坦白说&＃39;]

[&＃39;成绩&＃39;, &＃39;开心&＃39;, &＃39;朋友&＃39;]

[&＃39;疼爱&＃39;, &＃39;两年&＃39;, &＃39;相爱&＃39;]

[&＃39;记得&＃39;, &＃39;别怪&＃39;, &＃39;扣扣&＃39;]

[&＃39;其实&＃39;, &＃39;某人&＃39;, &＃39;呵呵&＃39;]

[&＃39;爸妈&＃39;, &＃39;玩电脑&＃39;, &＃39;干活&＃39;]

[&＃39;出轨&＃39;, &＃39;放不下&＃39;, &＃39;老公&＃39;]

[&＃39;好久不见&＃39;, &＃39;带著&＃39;, &＃39;需说&＃39;]

[&＃39;出轨&＃39;, &＃39;老公&＃39;, &＃39;可是&＃39;]

[&＃39;同桌&＃39;, &＃39;我们&＃39;, &＃39;有时候&＃39;]

[&＃39;出轨&＃39;, &＃39;分居&＃39;, &＃39;不知悔改&＃39;]

[&＃39;偷偷&＃39;, &＃39;见到&＃39;, &＃39;几面&＃39;]

[&＃39;哥哥&＃39;, &＃39;见到&＃39;, &＃39;那年&＃39;]

[&＃39;侮辱&＃39;, &＃39;出轨&＃39;, &＃39;那种&＃39;]

[&＃39;喜欢&＃39;, &＃39;豆子&＃39;, &＃39;炒豆&＃39;]

[&＃39;篮球队&＃39;, &＃39;朋友&＃39;, &＃39;说实&＃39;]

[&＃39;16&＃39;, &＃39;责怪&＃39;, &＃39;一点多&＃39;]

[&＃39;是不是&＃39;, &＃39;联系&＃39;, &＃39;心里&＃39;]

[&＃39;学校&＃39;, &＃39;正好&＃39;, &＃39;我们&＃39;]

[&＃39;再也不会&＃39;, &＃39;知道&＃39;, &＃39;委屈&＃39;]

[&＃39;很烦&＃39;, &＃39;分手&＃39;, &＃39;一起&＃39;]

[&＃39;我错&＃39;, &＃39;不想&＃39;, &＃39;分手&＃39;]

[&＃39;有的是&＃39;, &＃39;生活&＃39;, &＃39;潇洒风流&＃39;]

[&＃39;我怕&＃39;, &＃39;真的&＃39;, &＃39;对方&＃39;]

[&＃39;我怕&＃39;, &＃39;真的&＃39;, &＃39;对方&＃39;]

[&＃39;陪我玩&＃39;, &＃39;玩电脑&＃39;, &＃39;...........

[&＃39;姑娘&＃39;, &＃39;qq&＃39;, &＃39;哥哥&＃39;]

[&＃39;默默&＃39;, &＃39;一切&＃39;, &＃39;说出&＃39;]

[&＃39;..&＃39;, &＃39;朋友&＃39;, &＃39;一天&＃39;]

[&＃39;喜欢&＃39;, &＃39;永远&＃39;, &＃39;白杰&＃39;]

[&＃39;说真的&＃39;, &＃39;真的&＃39;, &＃39;知道&＃39;]

[&＃39;阿亲&＃39;, &＃39;事来&＃39;, &＃39;fuck&＃39;]

[&＃39;毛坤&＃39;, &＃39;沙县&＃39;, &＃39;九点&＃39;]

[&＃39;自己&＃39;, &＃39;因为&＃39;, &＃39;家人&＃39;]

可见&＃xff0c;jieba完胜了snownlp。

另外&＃xff0c;jieba提取关键词有两种不同的算法&＃xff0c;TF-IDF和TextRank&＃xff0c;因为我不要求高精度&＃xff0c;没有再测试了&＃xff0c;有兴趣的朋友可以自己测试。基于TextRank算法的代码如下&＃xff1a;

from jieba import analyse

# 引入TextRank关键词抽取接口

textrank &＃61; analyse.textrank

# 原始文本

text &＃61; "线程是程序执行时的最小单位&＃xff0c;它是进程的一个执行流&＃xff0c;\

是CPU调度和分派的基本单位&＃xff0c;一个进程可以由很多个线程组成&＃xff0c;\

线程间共享进程的所有资源&＃xff0c;每个线程有自己的堆栈和局部变量。\

线程由CPU独立调度执行&＃xff0c;在多CPU环境下就允许多个线程同时运行。\

同样多线程也可以实现并发操作&＃xff0c;每个请求分配一个线程来处理。"

print "\nkeywords by textrank:"

# 基于TextRank算法进行关键词抽取

keywords &＃61; textrank(text)

# 输出抽取出的关键词

for keyword in keywords:

print keyword &＃43; "/",

原载&＃xff1a;蜗牛博客

网址&＃xff1a;http://www.snailtoday.com

尊重版权&＃xff0c;转载时务必以链接形式注明作者和原始出处及本声明。