06-16-2014
Quote:
Originally Posted by
Corona688
If it's saving index.html, you forgot the --spider.
You can feed wget a list of URL's with awk '{...}' | wget -I - ...
I put the --spider but it says that still
so run wget withe the spider line
then again with it feeding into it?
like awk | wget?
or is that all just one command?
thanks!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'm running Fedora Core 6 as an FTP server on a powerMac G4...
I'm trying to create a script to remove files older than 3 days...
I'm able to find all data older than 3 days but it finds hidden files such as
/home/ftp/goossens/.canna
/home/ftp/goossens/.kde... (4 Replies)
Discussion started by: James_UK
4 Replies
2. Solaris
On Solaris, suppose there is a directory 'dir'.
Log files of size approx 1MB are continuously being
deposited here by scp command. I have a script that scans
this dir every 5 mins and moves away the log files that
have been deposited so far.
How do I design my script so that I pick up *only*... (6 Replies)
Discussion started by: sentak
6 Replies
3. UNIX for Advanced & Expert Users
I'm using wget 1.11.4 on Cygwin 1.5.25.
I'm trying to recursively download a directory tree, which is the root of a javadoc tree.
This is approximately the command line I tried:
wget -x -p -r http://<host>/.../apidoc
When it finished, it seemed like it downloaded... (0 Replies)
Discussion started by: dkarr
0 Replies
4. Shell Programming and Scripting
Can you tell me how to download the directory tree just starting from "project1/" in this URL?
"https://somesite.com/projects/t/project1/"
This command does not seem to do what I want as it downloads also files from the upper hierarchy:
wget --no-check-certificate --http-user=user... (4 Replies)
Discussion started by: majormark
4 Replies
5. Shell Programming and Scripting
Is there a way to customize ls to ignore files ending with ~ and #? (those are Emacs backup and auto-save files). I found -B option, which only ignores ~ files (2 Replies)
Discussion started by: yaroslavvb
2 Replies
6. Shell Programming and Scripting
Hello,
I know find can be prevented from recursing into directories with something like the following...
find . -name .svn -prune -a type d
But how can I completely prevent directories of a certain name (.svn) from being displayed at all, the top level and the children?
I really... (2 Replies)
Discussion started by: nwb123
2 Replies
7. Shell Programming and Scripting
Hello Unix Geeks,
I am in a situation to use wget for crawling a site where the site contains 5 IP addresses. Out of 5, 4 are accessible and 1 is having a problem due to firewall problems.
In this case, my wget is getting stuck with that X.X.X.X and giving up. How can I ignore this IP and... (4 Replies)
Discussion started by: sathyaonnuix
4 Replies
8. Shell Programming and Scripting
Dear All,
I am using find command
find /my_rep/*/RKYPROOF/*/*/WDM/HOME_INT/PWD_DATA -name rk*myguidelines*.pdf -print
The problem i am facing here is find /my_rep/*/
the directory after my_rep could be mice001, mice002 and mice001_PO, mice002_PO
i want to ignore mice***_PO directory... (3 Replies)
Discussion started by: yadavricky
3 Replies
9. Shell Programming and Scripting
i have a cron that mirrors a site periodically
wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0
i want to download this into a directory called /files
but when I do this, it always create a new directory called "123.456.789.0" (the hostname)
it puts it into /files/123.456.789.0
but... (3 Replies)
Discussion started by: vanessafan99
3 Replies
10. UNIX for Advanced & Expert Users
I am using aix. I would like to ignore the /u directory. I tried this but it is not working.
find / -type f -type d \( -path /u \) -prune -o -name '*rpm*' 2>/dev/null
/u/appx/ls.rpm
/u/arch/vim.rpm (4 Replies)
Discussion started by: cokedude
4 Replies
LEARN ABOUT DEBIAN
gkrellkam-list
GKRELLKAM-LIST(5) File Formats Manual GKRELLKAM-LIST(5)
NAME
gkrellkam-list - format of the list description files used by GKrellKam
DESCRIPTION
GKrellKam lists describe a series of image sources, used in order or at random by the GKrellKam plugin. Online lists (downloadable by
GKrellKam with HTTP) use this same syntax.
Blank lines and comments (beginning with '#') in GKrellKam lists are ignored. Every line of text that does not begin with a tab character
is considered an image source.
An image source can be one of several types:
image Sources of type image consist of a complete local filename to an image file.
url Type url is just that; a ftp:// or http:// URL pointing at an online image file.
script Type script is a system command, executed in a shell by GKrellKam when it's time to get this image. The output of the command is
assumed to be a complete filename of a local image file, optionally terminated with newlines/whitespace. The corresponding image is
loaded.
list Type list causes a different GKrellKam list to be included in the current one.
Each image source line should look like this:
type: name
The "type: " part of the line can be omitted for images and URLs, to make these list files backwards compatible with earlier versions of
GKrellKam, but it is suggested that you specify them. It will enhance the readability of the list file.
There are also properties that can be applied to types image, url, and script. Properties must follow the image source line that they mod-
ify, and must begin with a tab character. These include:
tooltip
This sets the message shown when the mouse is kept over the image panel in GKrellKam for a few seconds. If not set, the image's
filename or URL is shown instead.
seconds
This changes the number of seconds that an image will remain displayed. If not set, the "Default number of seconds" set for that
panel in the GKrellM configuration window will be used.
refresh
If set, this sets the minimum length of time that must elapse before the image is re-loaded. If the image cycles up before that
time, the old image will be used. For script types, this means that the script will not be re-run. For url types, the image will not
be fetched using wget. When this property is not set, url images will be downloaded every time they are displayed, and scripts will
be run every time.
EXAMPLE
This is a simple example of a GKrellKam list. For a better example, see example.list from this distribution.
# .krellkam.list
# This list rotates through ~/pics/mypic[1-4].jpg along with a
# webcam, and then displays the pictures in ~/lists/sub.list
image: /home/paul/pics/mypic1.jpg
/home/paul/pics/mypic2.jpg
image: /home/paul/pics/mypic3.jpg
http://www.usu.edu/webcam/fullsize.jpg
[tab] tooltip: This is the building where I work
image: /home/paul/pics/mypic4.jpg
[tab] seconds: 4
[tab] tooltip: Don't display this one very long
list: /home/paul/lists/sub.list
FILES
$HOME/.krellkam.list - The default source for the first GKrellKam panel
example.list - A sample GKrellKam list
AUTHOR
GKrellKam was written by paul cannon <paul@cannon.cs.usu.edu>. This manual page was written by the same author for inclusion in the GKrel-
lKam distribution, and may be used by others.
SEE ALSO
wget(8)
Dec 7, 2001 GKRELLKAM-LIST(5)