使用urllib.request构造http请求报错:ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]

使用下列代码抓取mrtzone.top网站报错:

import urllib.request

response = urllib.request.urlopen(‘https://www.mrtzone.top’)

print(response.read().decode(‘utf-8’))

报错信息:

Traceback (most recent call last):

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 1317, in do_open

    encode_chunked=req.has_header(‘Transfer-encoding’))

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 1229, in request

    self._send_request(method, url, body, headers, encode_chunked)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 1275, in _send_request

    self.endheaders(body, encode_chunked=encode_chunked)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 1224, in endheaders

    self._send_output(message_body, encode_chunked=encode_chunked)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 1016, in _send_output

    self.send(msg)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 956, in send

    self.connect()

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”, line 1392, in connect

    server_hostname=server_hostname)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py”, line 412, in wrap_socket

    session=session

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py”, line 853, in _create

    self.do_handshake()

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py”, line 1117, in do_handshake

    self._sslobj.do_handshake()

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

  File “/Users/tianmengxu/Desktop/web_crawlers/urlopen.py”, line 5, in <module>

    response = urllib.request.urlopen(‘https://www.mrtzone.top’)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 222, in urlopen

    return opener.open(url, data, timeout)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 525, in open

    response = self._open(req, data)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 543, in _open

    ‘_open’, req)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 503, in _call_chain

    result = func(*args)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 1360, in https_open

    context=self._context, check_hostname=self._check_hostname)

  File “/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py”, line 1319, in do_open

    raise URLError(err)

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>

[Finished in 0.9s with exit code 1]

[cmd: [‘/usr/local/bin/python3’, ‘-u’, ‘/Users/tianmengxu/Desktop/web_crawlers/urlopen.py’]]

[dir: /Users/tianmengxu/Desktop/web_crawlers]

[path: /Users/tianmengxu/.rvm/gems/ruby-2.6.3/bin:/Users/tianmengxu/.rvm/gems/ruby-2.6.3@global/bin:/Users/tianmengxu/.rvm/rubies/ruby-2.6.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/tianmengxu/.rvm/bin]

 

用urllib.request.open(url)只能访问一些http的网站,无法访问https网站

这是由于大部分https的网站需要有SSL证书,即使模拟了User-Agent访问,还是报错,https的安全基础是SSL,如果没有SSL证书,就会拒绝访问。

或者就是此次访问忽略证书验证,查了一下资料,具体实现如下:

需要导入一个模块:

import ssl

然后加入这段代码:

 ssl._create_default_https_context = ssl._create_unverified_context

这段代码可以取消证书的验证。

代码示例:

import urllib.request

import ssl

ssl._create_default_https_context = ssl._create_unverified_context 

response = urllib.request.urlopen(‘https://www.mrtzone.top’)

print(response.read().decode(‘utf-8’))

 加上橘色代码后问题解决。

 

未经允许不得转载:Mr.T的世界 » 使用urllib.request构造http请求报错:ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED]

赞 (0) 打赏

评论 0

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏