Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: For loop till the files downloaded
Top Forums Shell Programming and Scripting For loop till the files downloaded Post 302852493 by RudiC on Wednesday 11th of September 2013 04:29:00 PM
Old 09-11-2013
I guess you've got 80 files in your filename.txt. Why don't you create a temp file, appending every file name that wget fails with, and, with a while loop around your above for loop, retry reading that ever decreasing temp file? Like
Code:
cp filename.txt tempfile
while tempfile not empty
  do
  while read FILE
    do wget $FILE || echo $FILE >>tempfile2
    done < tempfile
  cp tempfile2 tempfile
  done

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

question regarding ftp. Files downloaded are of size Zero.

I need to download some files from a remote server using ftp. I have ftp'd into the site. I then do an mget * to retrieve all of the data files. Everything seems to proceed normally and I am given feedback that the files were downloaded. Now if I go into the DOS Shell or Windows explorer, it list... (5 Replies)
Discussion started by: ralphisnow
5 Replies

2. UNIX for Dummies Questions & Answers

Loop till you find a string in a fine <-- Need Help New to Unix Scripting

Guys - I am new to Unix scripting and am in need for a script that does the following. I have bits and pieces created and tested but i am just having a little difficult time getting it all together. - Loop through till it finds a string in a specific file. Any help is greatly appreciated. ... (1 Reply)
Discussion started by: mrehman
1 Replies

3. Shell Programming and Scripting

How to print lines till till a pattern is matched in loop

Dear All I have a file like this 112534554 446538656 444695656 225696966 226569744 228787874 113536566 443533535 222564552 115464656 225445345 225533234 I want to cut the file into different parts where the first two columns are '11' . The first two columns will be either... (3 Replies)
Discussion started by: anoopvraj
3 Replies

4. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

5. Shell Programming and Scripting

Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A... (10 Replies)
Discussion started by: o0110o
10 Replies

6. Web Development

php files are downloaded

Hello, I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser. Example: test.mydomain.com <-- php works 192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies

7. Shell Programming and Scripting

BASH scripting - Preventing wget messed downloaded files

hello. How can I detect within script, that the downloaded file had not a correct size. linux:~ # wget --limit-rate=20k --ignore-length -O /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm ... (6 Replies)
Discussion started by: jcdole
6 Replies

8. Shell Programming and Scripting

Grep with loop till search is done

I need help to put a script where it runs the svn command grep'ing for the ticket# in the comments to see if the ticket was used in the latest commit. so on command line: ./test.sh ticket-1 ticket-2 ticket-3 It should be able to check if ticket-1 is used first and if not then check if... (2 Replies)
Discussion started by: iaav
2 Replies

9. Shell Programming and Scripting

While loop till length of line is great enough

I have the following code: # Get the line of stations_info.txt starting with "${xstation1} " and copy it to file temp.txt grep "^${xstation1} " stations_info.txt > temp.txt # Get lat and long of station nl=0 ... (2 Replies)
Discussion started by: claire.a
2 Replies

10. Shell Programming and Scripting

Deleting multiple files off an ftp server once they have been downloaded

Hello, I have a server that I have to ftp files off and they all start SGRD and are followed by 6 numbers. SGRD000001 SGRD000002 SGRD000003 The script I have will run every 10 mins to pick up files as new ones will be coming in all the time and what I want to do is delete the files I have... (7 Replies)
Discussion started by: sph90457
7 Replies

LEARN ABOUT DEBIAN

dillo

dillo(1)							   USER COMMANDS							  dillo(1)

NAME

       dillo - web browser

SYNOPSIS

       dillo [OPTION]...  [--] [URL|FILE]...

DESCRIPTION

       Dillo  is a lightweight graphical web browser that aims to be secure.  It handles HTTP internally, and FILE, FTP, and DATA URIs are handled
       through a plugin system (dpi). In addition, INSECURE HTTPS support can be enabled. Both FTP and Dillo's download manager  use  the  wget(1)
       downloader.

       Dillo  displays	HTML,  text,  PNG, JPEG, and GIF files.  It handles cookies, HTTP authentication (basic and digest), proxying (basic), and
       some CSS.

       Framesets are displayed as links to frames, and there is currently no support for javascript or video.

OPTIONS

       -f, --fullwindow
	      Start in full window mode: hide address bar, navigation buttons, menu, and status bar.

       -g, --geometry GEO
	      Set initial window position where GEO is WxH[{+-}X{+-}Y].

       -h, --help
	      Display this help text and exit.

       -l, --local
	      Don't load images for these URL(s).

       -v, --version
	      Display version info and exit.

       -x, --xid XID
	      Open first Dillo window in an existing window whose window ID is XID.

EXIT STATUS

       0      No error.

       1      Internal error.

       2      Error in command line arguments.

ENVIRONMENT

       HOME (or HOMEDRIVE and HOMEPATH on Cygwin)
	      User's home directory.

       http_proxy
	      URL of proxy to send HTTP traffic through.

FILES

       dpid   Dillo plugin daemon

       dpidc  Control program for dpid.

       ~/.dillo/bm.txt
	      User bookmarks

       ~/.dillo/certs/
	      Saved certificates for HTTPS.

       ~/.dillo/cookies.txt
	      Stored cookies

       ~/.dillo/cookiesrc
	      Cookie settings

       ~/.dillo/dillorc
	      Configuration file.

       ~/.dillo/dpid_comm_keys
	      Keys used in dpi daemon communication.

       ~/.dillo/dpidrc
	      Contains name of directory containing dpis, and associates dpi files with protocols.

       ~/.dillo/keysrc
	      Keybindings.

       ~/.dillo/style.css
	      User style sheet

SEE ALSO

       wget(1)

       Dillo website: http://www.dillo.org

								 October 13, 2011							  dillo(1)
All times are GMT -4. The time now is 03:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy