11-01-2007
Personally I would fork/exec wget.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Experts I am new to UNIX shell programming ( or scripting). My problem is that
I have an ASCII file in which column wise data is present, the columns are seperated by spaces. I want to read each columns data and store it in arrays, next I will be using the arrays to perform some numerical... (1 Reply)
Discussion started by: FarhanNaseer
1 Replies
2. Shell Programming and Scripting
Hi All,
I am having trouble through, I am reading the input from tab delimited file containing several records,
e.g.
line1 field1 field2 field3 so on..
line2 field1 field2 field3 so on..
..
..
on the basis of certain fields for each record in input file, I have to retrieve... (1 Reply)
Discussion started by: Sonu4lov
1 Replies
3. SCO
I have a server with SCO Open Server 5 and I need to copy data files from the hard drive. Can this be completed with a Windows PC with the hard drive connected with a USB caddy? I have no other way to do this. Server not running.
TIA
Tim (2 Replies)
Discussion started by: taordw
2 Replies
4. Shell Programming and Scripting
I want help reading data from different position of a line
cat filename | cut -c120-131 6263-6280 or
cat filename | cut -c120-131 -c6263-6280 doesn't seem to work.
Please help
jak (2 Replies)
Discussion started by: jakSun8
2 Replies
5. Shell Programming and Scripting
Hi,
I have files that has got ebcdic character set and also, there are fields like binary and hex fields. is there a way to convert this to normal ascii data by taking care of comp & comp-3 fields?
Many Thanks!! (10 Replies)
Discussion started by: ahmedwaseem2000
10 Replies
6. Shell Programming and Scripting
Hi, I am writting some perl scripts for daily backup process. In which I want to pass some data/referance from another txt file. Text file contains only one column and multiple rows. I want to pass this data to variables of another perl script.
e.g.
Refdoc.txt file contains data as:
perl1... (3 Replies)
Discussion started by: n.dba
3 Replies
7. UNIX and Linux Applications
In LVM I have one volume grp and it has two logical volumes A and B and both are full,so I extended my volume grp to 1GB with another hard disk and add this space to A and put my resr data .Now problem is that I get data from first hard disk only but I m unable to find data which is in second hard... (0 Replies)
Discussion started by: Rahul chauhan
0 Replies
8. Shell Programming and Scripting
I have a file as below. It contains two data sets separated by >.
I want to pipe each data set to another program called psxy. How
can I get the different records
Have started doing as follows but it only passes the first data set
awk 'BEGIN {RS=">"};{print $0}' p.dat
cat p.dat... (12 Replies)
Discussion started by: kristinu
12 Replies
9. Programming
I have a list of files, 4 files for example
Then I have two buffers ifa and ifb giving me the file numbers
from which I need to get a group of data. For example, the first
four groups use file 1. The fifth group uses data from file 1 and
file 2.
I am thinking of storing all the data in the... (1 Reply)
Discussion started by: kristinu
1 Replies
10. Shell Programming and Scripting
I have a csv file called template.csv which has the following data
Name, Age, Height
Jessica Jesse, 18, 150
Now what I want to do is use a shell script to read the name age and height which looks like this:
#!bin/sh
INPUT='template.csv
while read Name Age Height
do
echo... (2 Replies)
Discussion started by: JSNY
2 Replies
LEARN ABOUT DEBIAN
httpindex
httpindex(1) General Commands Manual httpindex(1)
NAME
httpindex - HTTP front-end for SWISH++ indexer
SYNOPSIS
wget [ options ] URL... 2>&1 | httpindex [ options ]
DESCRIPTION
httpindex is a front-end for index++(1) to index files copied from remote servers using wget(1). The files (in a copy of the remote direc-
tory structure) can be kept, deleted, or replaced with their descriptions after indexing.
OPTIONS
wget Options
The wget(1) options that are required are: -A, -nv, -r, and -x; the ones that are highly recommended are: -l, -nh, -t, and -w. (See the
EXAMPLE.)
httpindex Options
httpindex accepts the same short options as index++(1) except for -H, -I, -l, -r, -S, and -V.
The following options are unique to httpindex:
-d Replace the text of local copies of retrieved files with their descriptions after they have been indexed. This is useful to display
file descriptions in search results without having to have complete copies of the remote files thus saving filesystem space. (See
the extract_description() function in WWW(3) for details about how descriptions are extracted.)
-D Delete the local copies of retrieved files after they have been indexed. This prevents your local filesystem from filling up with
copies of remote files.
EXAMPLE
To index all HTML and text files on a remote web server keeping descriptions locally:
wget -A html,txt -linf -t2 -rxnv -nh -w2 http://www.foo.com 2>&1 |
httpindex -d -e'html:*.html,text:*.txt'
Note that you need to redirect wget(1)'s output from standard error to standard output in order to pipe it to httpindex.
EXIT STATUS
Exits with a value of zero only if indexing completed sucessfully; non-zero otherwise.
CAVEATS
In addition to those for index++(1), httpindex does not correctly handle the use of multiple -e, -E, -m, or -M options (because the Perl
script uses the standard GetOpt::Std package for processing command-line options that doesn't). The last of any of those options ``wins.''
The work-around is to use multiple values for those options seperated by commas to a single one of those options. For example, if you want
to do:
httpindex -e'html:*.html' -e'text:*.txt'
do this instead:
httpindex -e'html:*.html,text:*.txt'
SEE ALSO
index++(1), wget(1), WWW(3)
AUTHOR
Paul J. Lucas <pauljlucas@mac.com>
SWISH++ August 2, 2005 httpindex(1)