Sponsored Content
Top Forums Shell Programming and Scripting Script to scrape page for and save data Post 302870171 by joeyg on Friday 1st of November 2013 10:09:42 AM
Old 11-01-2013
Not consistent with the mission of this help forum

What you are asking is for assistance in 'hacking' a website to extract all possible data from it. This is not in line with the goals of this website - to help people understand unix and develop good programming skills.

If you want a dataset, ask the company for the data.

We cannot be a source for providing programming to extract data in this manner.

If you wish to discuss this decision, kindly post to the section:
Post Here to Contact Site Administrators and Moderators
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to save as image from a web page

I used flot to create a graph and I would like to be able to save/export the graph as an image. In firefox on windows you can just ctl rt-click and you have a save as image feature (which I can automate with js) but...I need this to work on a linux browser. On linux in firefox I can print preview... (11 Replies)
Discussion started by: vincaStar
11 Replies

2. Shell Programming and Scripting

How to pass data from server (CGI script) to client (html page)

Hi I know how to pass data from client side (html file) to server using CGI script (POST method). I also know how to re-create the html page from server side after receiving the data (using printf). However I want to write static pages on client side (only the structure), and only to pass... (0 Replies)
Discussion started by: naamabm
0 Replies

3. Shell Programming and Scripting

Save page source, including javascript

I need to get the source code of a webpage. I have tried to use wget and curl, but it doesn't show the necessary javascript part of the source. I don't have to execute it, only to view the source. How do I do that? (1 Reply)
Discussion started by: locoroco
1 Replies

4. Shell Programming and Scripting

Get Permissions and save to data

Hi all; I have the following code which gives me kind of what I need: #!/usr/bin/perl use Fcntl ':mode'; # if ($ARGV ne "") { $filename = $ARGV; } else { print "Please specify a file!\n"; exit; } # if... (2 Replies)
Discussion started by: gvolpini
2 Replies

5. Shell Programming and Scripting

script for adding page number before page breaks

Hi, If there is an expert that can help: I have many txt files that are produced from pdftotext that include page breaks the page breaks seem to be unix style hex 0C. I want to add page numbers before each page break as in : Page XXXX Regards antman (9 Replies)
Discussion started by: antman
9 Replies

6. Shell Programming and Scripting

Open Page and save it using mozilla

HI Guys, I have one command which can open page and i want to save and exit from it. pf@home> mozilla 181.131.193.10/g/report.txt It will open one page now how can i save it. Thanks (1 Reply)
Discussion started by: pareshkp
1 Replies

7. Shell Programming and Scripting

Scrape 10 million pages and save the raw html data in mysql database

I have a list of 10 million page urls. I want those pages scraped and saved in the mysql database as raw html. I own a Linux VPS server with 1GB RAM and WHM/cPanel. I would like to scrape at least 100,000 urls in 24 hours. So can anyone give me some sample shell scripting code? (4 Replies)
Discussion started by: Viruthagiri
4 Replies

8. UNIX for Dummies Questions & Answers

Get a data and save

If I have a A.log 1 Air Flow Monitor : 34.070 Degrees C 2 Air Flow Monitor : 41.730 Degrees C 3 Air Flow Monitor : 35.340 Degrees C 4 Air Flow Monitor : 33.370 Degrees C 5 Air Flow Monitor : 36.770 Degrees C 6 Air Flow Monitor : 45.910 Degrees C 7 Air Flow Monitor ... (1 Reply)
Discussion started by: sabercats
1 Replies

9. Shell Programming and Scripting

Run sql query in shell script and output data save as delimited text

I want to run sql query in shell script and output data save as delimited text (delimited text would be comma) Code: SPOOL_FILE=/pgedw/dan.txt SQL=/pgedw/dan.sql sqlplus -s username/password@myhost:port/servicename <<EOF set head on set COLSEP , set linesize 32767 SET TRIMSPOOL ON SET... (8 Replies)
Discussion started by: Jaganjag
8 Replies
Gtk2::AboutDialog(3)					User Contributed Perl Documentation				      Gtk2::AboutDialog(3)

NAME
Gtk2::AboutDialog HIERARCHY
Glib::Object +----Glib::InitiallyUnowned +----Gtk2::Object +----Gtk2::Widget +----Gtk2::Container +----Gtk2::Bin +----Gtk2::Window +----Gtk2::Dialog +----Gtk2::AboutDialog INTERFACES
Glib::Object::_Unregistered::AtkImplementorIface Gtk2::Buildable METHODS
widget = Gtk2::AboutDialog->new list = $about->get_artists $about->set_artists ($artist1, ...) o $artist1 (string) o ... (list) list = $about->get_authors $about->set_authors ($author1, ...) o $author1 (string) o ... (list) string or undef = $about->get_comments $about->set_comments ($comments) o $comments (string or undef) string or undef = $about->get_copyright $about->set_copyright ($copyright) o $copyright (string or undef) list = $about->get_documenters $about->set_documenters ($documenter1, ...) o $documenter1 (string) o ... (list) Gtk2::AboutDialog->set_email_hook ($func, $data=undef) o $func (scalar) o $data (scalar) string or undef = $about->get_license $about->set_license ($license) o $license (string or undef) pixbuf or undef = $about->get_logo string or undef = $about->get_logo_icon_name $about->set_logo_icon_name ($icon_name) o $icon_name (string or undef) $about->set_logo ($logo) o $logo (Gtk2::Gdk::Pixbuf or undef) string or undef = $about->get_program_name $about->set_program_name ($name) o $name (string or undef) Gtk2->show_about_dialog ($parent, $first_property_name, ...) o $parent (Gtk2::Window or undef) o $first_property_name (string) o ... (list) the rest of a list of name=>property value pairs. This is a convenience function for showing an application's about box. The constructed dialog is associated with the parent window and reused for future invocations of this function. The dialog is shown nonmodally, and will be hidden by any response. string or undef = $about->get_translator_credits $about->set_translator_credits ($translator_credits) o $translator_credits (string or undef) Gtk2::AboutDialog->set_url_hook ($func, $data=undef) o $func (scalar) o $data (scalar) string or undef = $about->get_version $about->set_version ($version) o $version (string or undef) string or undef = $about->get_website string or undef = $about->get_website_label $about->set_website_label ($website_label) o $website_label (string or undef) $about->set_website ($website) o $website (string or undef) boolean = $about->get_wrap_license Since: gtk+ 2.8 $about->set_wrap_license ($wrap_license) o $wrap_license (boolean) Since: gtk+ 2.8 URL AND EMAIL HOOKS
When setting the website and email hooks for the Gtk2::AboutDialog widget, you should remember that the order is important: you should set the hook functions before setting the website and email URL properties, like this: $about_dialog->set_url_hook(&launch_web_browser); $about_dialog->set_website($app_website); otherwise the AboutDialog will not display the website and the email addresses as clickable. PROPERTIES
'artists' (Glib::Strv : readable / writable / private) List of people who have contributed artwork to the program 'authors' (Glib::Strv : readable / writable / private) List of authors of the program 'comments' (string : readable / writable / private) Comments about the program 'copyright' (string : readable / writable / private) Copyright information for the program 'documenters' (Glib::Strv : readable / writable / private) List of people documenting the program 'license' (string : readable / writable / private) The license of the program 'logo' (Gtk2::Gdk::Pixbuf : readable / writable / private) A logo for the about box. If this is not set, it defaults to gtk_window_get_default_icon_list() 'logo-icon-name' (string : readable / writable / private) A named icon to use as the logo for the about box. 'program-name' (string : readable / writable / private) The name of the program. If this is not set, it defaults to g_get_application_name() 'translator-credits' (string : readable / writable / private) Credits to the translators. This string should be marked as translatable 'version' (string : readable / writable / private) The version of the program 'website' (string : readable / writable / private) The URL for the link to the website of the program 'website-label' (string : readable / writable / private) The label for the link to the website of the program. If this is not set, it defaults to the URL 'wrap-license' (boolean : readable / writable / private) Whether to wrap the license text. SEE ALSO
Gtk2, Glib::Object, Glib::InitiallyUnowned, Gtk2::Object, Gtk2::Widget, Gtk2::Container, Gtk2::Bin, Gtk2::Window, Gtk2::Dialog COPYRIGHT
Copyright (C) 2003-2008 by the gtk2-perl team. This software is licensed under the LGPL. See Gtk2 for a full notice. perl v5.12.1 2010-07-05 Gtk2::AboutDialog(3)
All times are GMT -4. The time now is 11:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy