如何在Golang中为http.Get（）请求设置超时？

问题描述：

I'm making a URL fetcher in Go and have a list of URLs to fetch. I send http.Get() requests to each URL and obtain their response.

resp,fetch_err := http.Get(url)

How can I set a custom timeout for each Get request? (The default time is very long and that makes my fetcher really slow.) I want my fetcher to have a timeout of around 40-45 seconds after which it should return "request timed out" and move on to the next URL.

How can I achieve this?

我在Go中制作了URL提取程序，并具有要提取的URL列表。我向每个网址发送http.Get（） code>请求并获得其响应。 p>

  resp，fetch_err：= http.Get（url）
   code>  pre> 
 
 如何为每个Get请求设置自定义超时？  （默认时间很长，这会使我的提取程序真的很慢。）我希望提取程序的超时时间约为40-45秒，之后它应该返回“请求超时”并移至下一个URL。  p> 
 
 
如何实现？ p> 
  div>

答

Apparently in Go 1.3 http.Client has Timeout field

timeout := time.Duration(5 * time.Second)
client := http.Client{
    Timeout: timeout,
}
client.Get(url)

That's done the trick for me.

答

A quick and dirty way:

http.DefaultTransport.(*http.Transport).ResponseHeaderTimeout = time.Second * 45

This is mutating global state w/o any coordination. Yet it might be possibly okay for your url fetcher. Otherwise create a private instance of http.RoundTripper:

var myTransport http.RoundTripper = &http.Transport{
        Proxy:                 http.ProxyFromEnvironment,
        ResponseHeaderTimeout: time.Second * 45,
}

var myClient = &http.Client{Transport: myTransport}

resp, err := myClient.Get(url)
...

Nothing above was tested.

答

You need to set up your own Client with your own Transport which uses a custom Dial function which wraps around DialTimeout.

Something like (completely untested) this:

var timeout = time.Duration(2 * time.Second)

func dialTimeout(network, addr string) (net.Conn, error) {
    return net.DialTimeout(network, addr, timeout)
}

func main() {
    transport := http.Transport{
        Dial: dialTimeout,
    }

    client := http.Client{
        Transport: &transport,
    }

    resp, err := client.Get("http://some.url")
}

答

To add to Volker's answer, if you would also like to set the read/write timeout in addition to the connect timeout you can do something like the following

package httpclient

import (
    "net"
    "net/http"
    "time"
)

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error) {
    return func(netw, addr string) (net.Conn, error) {
        conn, err := net.DialTimeout(netw, addr, cTimeout)
        if err != nil {
            return nil, err
        }
        conn.SetDeadline(time.Now().Add(rwTimeout))
        return conn, nil
    }
}

func NewTimeoutClient(connectTimeout time.Duration, readWriteTimeout time.Duration) *http.Client {

    return &http.Client{
        Transport: &http.Transport{
            Dial: TimeoutDialer(connectTimeout, readWriteTimeout),
        },
    }
}

This code is tested and is working in production. The full gist with tests is available here https://gist.github.com/dmichael/5710968

Be aware that you will need to create a new client for each request because of the conn.SetDeadline which references a point in the future from time.Now()

答

You may use https://github.com/franela/goreq which handles timeouts in a fashion and simple way.

答

If you want to do it per request, err handling ignored for brevity:

ctx, cncl := context.WithTimeout(context.Background(), time.Second*3)
defer cncl()

req, _ := http.NewRequest(http.MethodGet, "https://google.com", nil)

resp, _ := http.DefaultClient.Do(req.WithContext(ctx))

答

timeout := time.Duration(5 * time.Second)
transport := &http.Transport{Proxy: http.ProxyURL(proxyUrl), ResponseHeaderTimeout:timeout}

This may help, but notice that ResponseHeaderTimeout starts only after the connection is established.

如何在Golang中为http.Get（）请求设置超时？

相关推荐