Shell Programming and Scripting

Now,

I have downloaded as suggested by Alister. My files are named like this

Code:

index.html?acc=OSR765454&file=filename1.gz
index.html?acc=OBR765454&file=filename111.gz
index.html?acc=ORS765454&file=filename1111.gz
index.html?acc=OST76454&file=filename11111.gz
index.html?acc=OS5454&file=filename1111111.gz

I was using the following command

Code:

mv index.html?acc=OSR765454&file=filename1.gz filename1.gz

The error I see is the following

Code:

[1] 14709
mv: missing destination file operand after `index.html?acc=OSR765454'
Try `mv --help' for more information.
-bash: filename1.gz: command not found
[1]+  Exit 1                  mv index.html?acc=OSR765454

Any thoughts?

The & is a special character to the shell. The command is cut off at that point. So the shell takes everything before it as a command to run in the background.

Code:

[1]+  Exit 1                  mv index.html?acc=OSR765454Exit 11

And then afterwards will run the following command:

Code:

file=filename1.gz filename1.gz

Which creates a variable named file with a value of filename1.gz intended for the environment of the command filename1.gz (which of course doesn't exist and so you get that command not found error).

To fix that mv command, quote any arguments with shell metacharacters. In your example, aside from the &, the ? is also special.

Don't blame this on me. I never suggested doing this. I merely mentioned that you could try using rename if you insisted on using wget -i. I suggested keeping it simple with scott's solution. I don't understand why you don't like that simple while-loop. Did you run it? If so, was it unacceptably slow? Did it give you an erroneous result?

Regards,
Alister

Alister,

I do love Scott's solution, but the point is after I get a bunch of those wget commands, I have to run them individually.

But, anyways I used the following command

Code:

mv index.html'?'acc=OSR765454'&'file=filename1.gz filename1.gz

And it works like a charm. Do you have a simple solution to do the above task in batch on all the files in the directory using rename? Because, I can't write the mv command for all those files in my directory. They are around 400 files. And the OSR number after acc= is varying. Can you suggest me something using sed and rename or sed and mv?

Thanks for ur time.

Why manually? In your original post you stated that you had the list of urls in a file. Is that not the case? Are you not now using that file as an option-argument thusly:
If so, just use this instead (I took the liberty of adding -r and quotes to scott's code):
It's just as automated.

Or have I missed something?

Regards,
Alister

Alister,

You missed a point.

I did check my output files after editing the input.txt to the following

Code:

index.html?acc=OSR765454&file=filename1.gz -O filename1.gz
index.html?acc=OBR765454&file=filename111.gz -O filename111.gz
.....

This input.txt has to be given to the command.

I did that and all I see is a single output file instead of one. If all the 5 file sizes together is 10GB. I see a single file filename1.gz to be 10GB.

Hope you got my point.

Scott's solution does not use -i and is intended to work with the input.txt as initially specified:
Did you try his while-loop with that input data? It should work and no file renaming should need to be done.

Regards,
Alister