爬取微博评论时找不到https://m.weibo.cn/api/comments/show?id=

"""

爬取微博评论&＃xff0c;保存到数据库

 

https://m.weibo.cn/api/comments/show?id&＃061;4477013081328252&page&＃061;50

该接口能获取微博的前50页数据&＃xff0c;每页10条&＃xff0c; id 是某条微博的id

 

https://m.weibo.cn/comments/hotflow?mid&＃061;4477013081328252&max_id&＃061;330569188932643&max_id_type&＃061;0

此接口能爬到所有评论信息&＃xff0c; mid 是某条微博id&＃xff0c; max_id 是上一个请求返回的分页参数&＃xff0c; max_id_type 固定为0就好

"""

 

from pymongo import MongoClient

import requests

import time

 

__author__ &＃061; &＃039;liuzhijun&＃039;

 

headers &＃061; {

    "Host": "m.weibo.cn",

    "User-Agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) "

                  "Version/9.0 Mobile/13B143 Safari/601.1",

    "COOKIE": "xxxxx" # 这里将浏览器的COOKIE复制过来进行了。

}

 

client &＃061; MongoClient(&＃039;mongodb://localhost:27017/&＃039;)

db &＃061; client[&＃039;weibo&＃039;]

 

def main(mid, max_id):

    """

    :param mid: 某条微博id

    :param max_id: 分页参数

    :return:

    """

    url &＃061; "https://m.weibo.cn/comments/hotflow?max_id_type&＃061;0"

    params &＃061; {"mid": mid}

    if max_id:

        params[&＃039;max_id&＃039;] &＃061; max_id

 

    res &＃061; requests.get(url, params&＃061;params, headers&＃061;headers)

    print(res.content)

    result &＃061; res.json()

    max_id &＃061; result.get("data").get("max_id")

    data &＃061; result.get(&＃039;data&＃039;).get(&＃039;data&＃039;)

    for item in data:

        db[&＃039;comment&＃039;].insert_one(item)

 

    if max_id:

        time.sleep(1)

        main(mid, max_id)

 

if __name__ &＃061;&＃061; &＃039;__main__&＃039;:

    main("4477013081328252", None)