Collecting Data from Website
1 posts in topic
Flat View  Flat View
Older Post First |   Reply to Topic
TOPIC ACTIONS:
 

Posted By:   Trupti_Mehta
Posted On:   Monday, June 19, 2006 06:12 AM

How to collect data from a website? I understand, this is not a RMI question, I didn't get which forum would be the best to question this. Sorry for inconvinience. Hi Friends, I am mainly an Application Developer. I have worked with Web & Wap also. Nowadays, I see everywhere, retriving data from website and working/manipulating it. Like, many asks to get data like email addresses, customers info, etc from websites. Websites may be coded with Java or some other lang. Something more specific, I read the below details, and am curious to know the procedure to accomplish this :- * read/collect daily market data from Stock Quotation software(JAVA or others) *) automatio   More>>

How to collect data from a website?


I understand, this is not a RMI question, I didn't get which forum would be the best to question this. Sorry for inconvinience.



Hi Friends,


I am mainly an Application Developer. I have worked with Web & Wap also. Nowadays, I see everywhere, retriving data from website and working/manipulating it. Like, many asks to get data like email addresses, customers info, etc from websites. Websites may be coded with Java or some other lang. Something more specific, I read the below details, and am curious to know the procedure to accomplish this :-


* read/collect daily market data from Stock Quotation software(JAVA or others)


*) automation of trading instructions by manipulating client Online Stock Trading software(coded in other or JAVA), this may involve decompiling java/other quotation software.


*) organise daily quotation and trading data into the database and should be able to make simple report and analysis

I would like to know, how can we accomplish these kind of tasks. Also, if u r aware of useful tutorials on such topics will be great. Any help is greatly appreciated.


Thanks



Trupti

   <<Less
Report | Quote This | Send private message | Reply | Print

Re: Collecting Data from Website

Posted By:   Anonymous  
Posted On:   Monday, June 19, 2006 03:41 PM

Hi Trupti,

You can accomplish using apache commons http client.
See the sample program. Using this you can get the page. You have to parse after this, to get the required info.



HttpClient httpClient = new HttpClient();

httpClient.setConnectionTimeout(5000);

HttpMethod method = new GetMethod(pageURL);

method.setFollowRedirects(true);

int responseCode = httpClient.executeMethod(method);

if (responseCode == 200) {

String pageData = method.getResponseBodyAsString();

}
Report | Quote This | Send private message | Reply | Print
About | Sitemap | Contact