Reading data from a website

Good day, I have been tasked to get temperature data on a daily basis from a website.  I thought that I had seen something about this from somewhere, but can't seem to put my finger on it, so, Is there a way that I can automatically get data from another website that is stored in a table, by calling a function...?  I want to get two values from specific days. I would like to be able to take all the data that is stored in a table and put it into a structure then I can parse out what I need.   so A) Is this possible?    B) a Few pointers if you don't mind? Thanks, Rob

> I have been tasked to get temperature data on a daily basis from a website Good evening, This very much depends on a couple of factors: * If you own the application that is publishing the temperatures -  You could create a webservice that other applications can access, -  You could publish rss feeds that other applications can access * If you do not own the application that is publishing the temperatures: - If by table you mean HTML table, then it *should* be possible to parse the html and extract the data you need though this depends on the html source. Post a link and I'll let you know ;) - If by table you mean database table, then unless the application formally publishes the data through a feed or webservice then you won't be able to access it. HTH Dominic -- Blog it up: http://fusion.dominicwatson.co.uk

Good evening to you Dominic, below is the link to the website that contains the data I require. It is in an HTML table, so I guess that I am screen scraping? http://www.climate.weatheroffice.ec.gc.ca/climateData/dailydata_e.html?timeframe=2&Prov=CA&StationID=43763&Year=2008&Month=4&Day=17 I have the same page for 5 other locations. There are problems with using a webservice etc, as this is for an agricultural business and the locations that I require information for smaller locations.  I did not find a webservice that even carried Canadian data :). TIA, Rob ----- Excess quoted text cut - see Original Post for more -----

Right, parsing that page, as Dave said, will suck. It's not xml so xml parsing is out (that would then be reasonably easy). So you'd need a load of regex and parsing wizardry. It IS possible though. Again, as Dave pointed out, when their code changes, your app breaks. However, if you look at the navigation options in the bottom right hand corner you will see a Bulk data link with a link to a page that generates Xml of the data :) Have a look at some of that xml and you will find all the data ready for simple xml parsing; it looks like you can get data for months, whole years, etc - as it is built dynamically from a URL query string. Have fun with that! Dominic -- Blog it up: http://fusion.dominicwatson.co.uk

>>below is the link to the website that contains the data I require. Ok, you definitely need CF_REextract: http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm You can even test it on your page here: http://www.contentbox.com/claude/customtags/REextract/testingREextract.cfm 1. Select http for inputmode 2. Enter dataTableRowData"> in RE1 3. enter </td> in RE2 4. enter the address of your page in the URL field, 5. click Test, and voilààààà... For your final solution, you will use a just little more sophisticated loop to extract exactly what you need. See also this page which uses the same tag to extract weather forecast from environment Canada: http://www.fafo.on.ca/ -- _______________________________________ REUSE CODE! Use custom tags; See http://www.contentbox.com/claude/customtags/tagstore.cfm (Please send any spam to this address: piegeacon@internetique.com) Thanks.

Once again, you "definitely" DON'T need it. Paul has already offered two MUCH more effiecient FREE solutions. .:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com http://cf4em.com >>below is the link to the website that contains the data I require. Ok, you definitely need CF_REextract: http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm You can even test it on your page here: http://www.contentbox.com/claude/customtags/REextract/testingREextract.cfm 1. Select http for inputmode 2. Enter dataTableRowData"> in RE1 3. enter </td> in RE2 4. enter the address of your page in the URL field, 5. click Test, and voil?????... For your final solution, you will use a just little more sophisticated loop to extract exactly what you need. See also this page which uses the same tag to extract weather forecast from environment Canada: http://www.fafo.on.ca/ -- _______________________________________ REUSE CODE! Use custom tags; See http://www.contentbox.com/claude/customtags/tagstore.cfm (Please send any spam to this address: piegeacon@internetique.com) Thanks.

>  >>below is the link to the website that contains the data I require. > >  Ok, you definitely need CF_REextract: >  http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm No, you don't. I have NEVER come across a situation where I needed a commercial solution to regex in CF, and I use regex a lot and have done screen scraping with it too. -- Jay

On Tue, Apr 22, 2008 at 7:31 AM, James Smith <james@music-express.co.uk> wrote: ----- Excess quoted text cut - see Original Post for more ----- http://tinyurl.com/5holdg -- Evelyn the dog, having undergone further modification pondered the significance of short-person behaviour in pedal depressed, pan-chromatic resonance, and other highly ambient domains. "Arf," she said.

I'll take one! :-) .:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com http://cf4em.com http://tinyurl.com/5holdg

On Mon, Apr 21, 2008 at 1:57 PM, Rob Parkhill <robert.parkhill@gmail.com> wrote: > Good day, > >  I have been tasked to get temperature data on a daily basis from a website. I thought that I had seen something about this from somewhere, but can't seem to put my finger on it, so, Is there a way that I can automatically get data from another website that is stored in a table, by calling a function...?  I want to get two values from specific days. > >  I would like to be able to take all the data that is stored in a table and put it into a structure then I can parse out what I need. http://www.google.com/search?q=weather+webservice that'd be a place to start.  find the web service that suits your needs.  depending on which one you choose, there may even be docs available that show how to integrate it into your site. -- Evelyn the dog, having undergone further modification pondered the significance of short-person behaviour in pedal depressed, pan-chromatic resonance, and other highly ambient domains. "Arf," she said.

----- Excess quoted text cut - see Original Post for more ----- If the web site has a web service - a program you can call directly via a HTTP URL - then, yes, you can easily get whatever data that web service publishes. Web services typically publish data using XML, which is easily parsed. Otherwise, you will have to use CFHTTP to fetch the web page in question, and you will have to use string functions to parse out the bits you need, and the process of building and maintaining this functionality will suck, to put it bluntly. If the formatting of the web page changes later, you'll probably have to edit your code accordingly. This process is generally referred to as "screen scraping". I would go for the web service approach. There are probably plenty of web sites that offer a daily temperature web service, although they might want you to pay for it. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Training: Adobe/Google/Paperthin Certified Partners http://training.figleaf.com/ WebManiacs 2008: the ultimate conference for CF/Flex/AIR developers! http://www.webmaniacsconference.com/

http://tutorial451.easycfm.com/ :) ----- Excess quoted text cut - see Original Post for more -----