python - HTTP 错误 503 服务不可用
我正在尝试从 google 和linkedin 中抓取数据.不知何故，它给了我这个错误:
I am trying to scrape data from google and linkedin. Somehow it gave me this error:
*** httperror_seek_wrapper: HTTP Error 503: Service Unavailable
有人可以帮忙建议我如何解决这个问题吗?
Can someone help advice how I solve this?
我遇到了同样的情况，并尝试在每次请求之前使用 sleep() 函数将请求分散一点.看起来它工作正常，但即使延迟 2 秒也很快失败.最终解决的是使用:
I encountered the same situation and tried using the sleep() function before every request to spread the requests a little. It looked like it was working fine but failed soon enough even with a delay of 2 seconds. What solved it finally was using:
with contextlib.closing(urllib.urlopen(urlToOpen)) as x:
#do stuff with x.
我这样做是因为我认为打开太多请求会使它保持打开状态而不得不关闭.尽管如此，它的工作非常一致，延迟时间低至 0.5 秒.
This I did because I thought opening too many requests keeps it open and had to closed. Nevertheless, it worked quite consistently with as less as 0.5s delay time.