Sponsored Content
Top Forums Shell Programming and Scripting For loop till the files downloaded Post 302852493 by RudiC on Wednesday 11th of September 2013 04:29:00 PM
Old 09-11-2013
I guess you've got 80 files in your filename.txt. Why don't you create a temp file, appending every file name that wget fails with, and, with a while loop around your above for loop, retry reading that ever decreasing temp file? Like
Code:
cp filename.txt tempfile
while tempfile not empty
  do
  while read FILE
    do wget $FILE || echo $FILE >>tempfile2
    done < tempfile
  cp tempfile2 tempfile
  done

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

question regarding ftp. Files downloaded are of size Zero.

I need to download some files from a remote server using ftp. I have ftp'd into the site. I then do an mget * to retrieve all of the data files. Everything seems to proceed normally and I am given feedback that the files were downloaded. Now if I go into the DOS Shell or Windows explorer, it list... (5 Replies)
Discussion started by: ralphisnow
5 Replies

2. UNIX for Dummies Questions & Answers

Loop till you find a string in a fine <-- Need Help New to Unix Scripting

Guys - I am new to Unix scripting and am in need for a script that does the following. I have bits and pieces created and tested but i am just having a little difficult time getting it all together. - Loop through till it finds a string in a specific file. Any help is greatly appreciated. ... (1 Reply)
Discussion started by: mrehman
1 Replies

3. Shell Programming and Scripting

How to print lines till till a pattern is matched in loop

Dear All I have a file like this 112534554 446538656 444695656 225696966 226569744 228787874 113536566 443533535 222564552 115464656 225445345 225533234 I want to cut the file into different parts where the first two columns are '11' . The first two columns will be either... (3 Replies)
Discussion started by: anoopvraj
3 Replies

4. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

5. Shell Programming and Scripting

Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A... (10 Replies)
Discussion started by: o0110o
10 Replies

6. Web Development

php files are downloaded

Hello, I have setup Cherokee web server and php 5.2 in Opensolaris zone. Problem is that all .php files are downloaded from web server and not served when I use IP address instead of DNS name in web brovser. Example: test.mydomain.com <-- php works 192.168.0.10/index.php <--... (3 Replies)
Discussion started by: kreno
3 Replies

7. Shell Programming and Scripting

BASH scripting - Preventing wget messed downloaded files

hello. How can I detect within script, that the downloaded file had not a correct size. linux:~ # wget --limit-rate=20k --ignore-length -O /Software_Downloaded/MULTIMEDIA_ADDON/skype-4.1.0.20-suse.i586.rpm ... (6 Replies)
Discussion started by: jcdole
6 Replies

8. Shell Programming and Scripting

Grep with loop till search is done

I need help to put a script where it runs the svn command grep'ing for the ticket# in the comments to see if the ticket was used in the latest commit. so on command line: ./test.sh ticket-1 ticket-2 ticket-3 It should be able to check if ticket-1 is used first and if not then check if... (2 Replies)
Discussion started by: iaav
2 Replies

9. Shell Programming and Scripting

While loop till length of line is great enough

I have the following code: # Get the line of stations_info.txt starting with "${xstation1} " and copy it to file temp.txt grep "^${xstation1} " stations_info.txt > temp.txt # Get lat and long of station nl=0 ... (2 Replies)
Discussion started by: claire.a
2 Replies

10. Shell Programming and Scripting

Deleting multiple files off an ftp server once they have been downloaded

Hello, I have a server that I have to ftp files off and they all start SGRD and are followed by 6 numbers. SGRD000001 SGRD000002 SGRD000003 The script I have will run every 10 mins to pick up files as new ones will be coming in all the time and what I want to do is delete the files I have... (7 Replies)
Discussion started by: sph90457
7 Replies
All times are GMT -4. The time now is 02:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy