11-01-2013
Script to scrape page for and save data
Hi All
I would like to get cars DB from this web site link removed , and I am trying to right script to go and parse the web page and save the data for each cars into a file, type of cars, mark, model, but when I look to the source page I found out that the first car type is preselected and the car marks also, if I want the models of cars I have the change the selected mark on the dropdown and then the models is populated, I think this function is done by Javascript but I can't find it in the source code. it will be good if you help me out to right script that change car mark, save the information cars type, marks, models into a file.
Doing this manually will be a lot of work.
Thanks.
Last edited by joeyg; 11-01-2013 at 08:33 AM..
9 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I used flot to create a graph and I would like to be able to save/export the graph as an image. In firefox on windows you can just ctl rt-click and you have a save as image feature (which I can automate with js) but...I need this to work on a linux browser. On linux in firefox I can print preview... (11 Replies)
Discussion started by: vincaStar
11 Replies
2. Shell Programming and Scripting
Hi
I know how to pass data from client side (html file) to server using CGI script (POST method).
I also know how to re-create the html page from server side after receiving the data (using printf).
However I want to write static pages on client side (only the structure), and only to pass... (0 Replies)
Discussion started by: naamabm
0 Replies
3. Shell Programming and Scripting
I need to get the source code of a webpage. I have tried to use wget and curl, but it doesn't show the necessary javascript part of the source. I don't have to execute it, only to view the source.
How do I do that? (1 Reply)
Discussion started by: locoroco
1 Replies
4. Shell Programming and Scripting
Hi all;
I have the following code which gives me kind of what I need:
#!/usr/bin/perl
use Fcntl ':mode';
#
if ($ARGV ne "") {
$filename = $ARGV;
} else {
print "Please specify a file!\n";
exit;
}
#
if... (2 Replies)
Discussion started by: gvolpini
2 Replies
5. Shell Programming and Scripting
Hi,
If there is an expert that can help:
I have many txt files that are produced from pdftotext that include page breaks the page breaks seem to be unix style hex 0C.
I want to add page numbers before each page break as in : Page XXXX
Regards antman (9 Replies)
Discussion started by: antman
9 Replies
6. Shell Programming and Scripting
HI Guys,
I have one command which can open page and i want to save and exit from it.
pf@home> mozilla 181.131.193.10/g/report.txt
It will open one page now how can i save it.
Thanks (1 Reply)
Discussion started by: pareshkp
1 Replies
7. Shell Programming and Scripting
I have a list of 10 million page urls. I want those pages scraped and saved in the mysql database as raw html.
I own a Linux VPS server with 1GB RAM and WHM/cPanel.
I would like to scrape at least 100,000 urls in 24 hours.
So can anyone give me some sample shell scripting code? (4 Replies)
Discussion started by: Viruthagiri
4 Replies
8. UNIX for Dummies Questions & Answers
If I have a A.log
1 Air Flow Monitor : 34.070 Degrees C
2 Air Flow Monitor : 41.730 Degrees C
3 Air Flow Monitor : 35.340 Degrees C
4 Air Flow Monitor : 33.370 Degrees C
5 Air Flow Monitor : 36.770 Degrees C
6 Air Flow Monitor : 45.910 Degrees C
7 Air Flow Monitor ... (1 Reply)
Discussion started by: sabercats
1 Replies
9. Shell Programming and Scripting
I want to run sql query in shell script and output data save as delimited text (delimited text would be comma)
Code:
SPOOL_FILE=/pgedw/dan.txt
SQL=/pgedw/dan.sql
sqlplus -s username/password@myhost:port/servicename <<EOF
set head on
set COLSEP ,
set linesize 32767
SET TRIMSPOOL ON
SET... (8 Replies)
Discussion started by: Jaganjag
8 Replies
car(1) GNU Telephony car(1)
NAME
car - crytographic archiver.
SYNOPSIS
car [options] [paths...]
car --decode [.carfile]
DESCRIPTION
Creates and decodes portable cross-platform crytographic archives. An archive can be a collection of files, or an in-stream message that
is piped. Output can be to a binary .car file, or ascified text. A symetric cipher is used, and the passhrase is hashed to form a key.
OPTIONS
--cipher=name
Specify symetric cipher. By default 256 bit aes is used.
--decrypt
Specify decryption operation on an existing car stream or file. If no file is specified, stdin is used.
--digest=name
Specify name of digest algorithm. By default sha will be used.
--follow
Dereference and follow symlinks. Otherwise they are ignored.
--output=filename
Specify output file for a new archive. By default stdout is used.
--overwrite
Always overwrite existing files without prompting.
--quiet
Non-interactive and no status output.
--recursive
If argument is a directory, recursively scan directory and any subdirectory contents as arguments.
--help Outputs help screen for the user.
AUTHOR
car was written by David Sugar <dyfet@gnutelephony.org>.
REPORTING BUGS
Report bugs to bug-commoncpp@gnu.org.
COPYRIGHT
Copyright (C) 2010 David Sugar, Tycho Softworks.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
LAR PURPOSE.
GNU uCommon January 2010 car(1)