Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Store data from dynamic website table

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Store data from dynamic website table
# 1  
Old 04-12-2010
Store data from dynamic website table
hi everybody,


Asking for something that I´m not sure if it´s possible to implement. I hope be clear enough.

Well, my issue is that I´m looking how to copy or extract a particular table content of a website. I get the content from a external feed (Iframe format), the content is updated every minute. The Iframe updates new results/scores if there are new ones to show, and for instance, could show from 20 to 100 lines in the table in average at the same time.

Since the information showed by the feed is dynamic, I can´t look for old results(e.g. last week, month, etc) Thus, my goal, (if it´s possible) is to have a unix script (the server runs in unix) that reads the website
periodically (e.g twice a day) and copies in a MySQL database or txt file what table feed shows, within the hosting server. With this would be possible build a history of results and be able to see old results
of desired event in any moment when visitors want.

The IFrame table shows something like this:

Code:
Country 1: Place
04-12-2010 20:45 - cloudy - partly cloudy  Now    
Country 2: Place
04-12-2010 19:00 - cloudy - cloudy  Now
.
.
.
Country N: Place
04-12-2010 17:30 - Sunny - Partly cloudy  Now

Many thanks for any advice or help with this.

Best regards.

---------- Post updated at 07:07 PM ---------- Previous update was at 02:57 AM ----------

Hi again,

I think is no the right section to this thread.

Moderator, may you please move my thread to Web Development section please?

Thanks in advance for any help or suggestion about my question.

Regards.
# 2  
Old 04-13-2010
It is possible to 'scrap' from web pages. See some of my old blogs

Chi Hung Chan
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help Needed: UNIX shell variables to store Oracle table records

Hello Folks, I'm working on a requirement to automate the process of generating report(csv file) using metadata info stored in an Oracle table and E-mail it to respective people. Meta data table: Report_ID,Report_SUB_ID,Report_DB,Report_SQL,Report_to_email_Id 1,1,DEV,'select * From... (2 Replies)
Discussion started by: venkat_reddy
2 Replies

2. Shell Programming and Scripting

Store results in table

Hello everyone, I have a shell script, which connects to the database and runs .sql file. after executing of .sql file, i need to store the results in error table. How can i achieve this one? could you please give your suggestions. here is my code. #!/bin/sh #set -vx SCHEMA_NAME=$1... (1 Reply)
Discussion started by: Rami Reddy
1 Replies

3. Web Development

MYSQL: Creating Dynamic Table Names 5.1

Hey everyone. Thanks for looking at this. I'm trying to create a table with the dynamic name of TableName + today's date. My variables are all happily created but the system chokes when I try to create the new table name example: Set @BFBW = CONCAT("BFBW", CURDATE()); Select @BFBW; ... (2 Replies)
Discussion started by: Astrocloud
2 Replies

4. Shell Programming and Scripting

Script to store the variable in a table or array.

Hi, I have few variable say 10 ex:- l_pc_291334_01_0_01_00.cmp l_pc_441133_50_0_02_00.cmp l_pc_441133_55_0_02_00.cmp Each variable value is coming via loop on a table. I want to create a script that stores these value to a table or array ( But one by one not all at one time as... (4 Replies)
Discussion started by: amitkumar.b2
4 Replies

5. Shell Programming and Scripting

Store table contents in csv file

I need to write a script to store the contents of a table in a csv file I'm using Toad, it's a Oracle database. (5 Replies)
Discussion started by: ladyAnne
5 Replies

6. Shell Programming and Scripting

Using Python to grab data from a website

Hello Everyone, I'm trying to write a python script that will go to the following website and grab all the data on the page. The page refreshes regularly and the number of flights is different. Untitled Document What I wanted to do was grab all the data (except for top three row containing... (5 Replies)
Discussion started by: jl487
5 Replies

7. UNIX for Advanced & Expert Users

Sql dynamic table / dynamic inserts

I have a file that reads File (X.txt) Contents of record 1: rdrDESTINATION_ADDRESS (String) "91 971502573813" rdrDESTINATION_IMSI (String) "000000000000000" rdrORIGINATING_ADDRESS (String) "d0 movies" rdrORIGINATING_IMSI (String) "000000000000000" rdrTRAFFIC_EVENT_TIME... (0 Replies)
Discussion started by: magedfawzy
0 Replies

8. Shell Programming and Scripting

store the table data in excel file

Hello - I have a below table and i want to extract the data into excel sheet and send to different location. Here is the table structure... SQL> desc t_i1_exportdocs Name Null? Type ----------------------------------------- --------... (11 Replies)
Discussion started by: govindts
11 Replies

9. AIX

Dynamic routing table

i have AIX Unix with "dynamic routing table" service enabled, but i do not need this service. How i can disable this service. thanks (1 Reply)
Discussion started by: jdsnbr
1 Replies

10. Shell Programming and Scripting

Store return code of shell script in oracle table

Hi friends, I have to do the following things : 1) there should be a shell script returning the returning the return code of the script. and i have to add some more details like on which machine is has run , at what time and other details and then using plsql i have to add a row to Oracle... (3 Replies)
Discussion started by: sveera
3 Replies
Login or Register to Ask a Question