I have around 5000 URL links. I want to check the HTTP status of each link.

Christopher Pickslay

I was doing something very similar, and ran into the same problem. In general, it's more efficient to run several threads simultaneously, each checking a URL from your list. And you can set a timeout on each thread so it gives up after a given period.

Also, if all you care about is the response code, try setting the request method to "HEAD" using connection.setRequestMethod("HEAD"). This will cause the server to return only the headers, and not the full body of the HTML document, which you're ignoring anyways. This will save you bandwidth, and your URL checker will run a lot faster.

0 Comments  (click to add your comment)
Comment and Contribute

 

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

About | Sitemap | Contact