Bash to tell download where specific files are stored


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to tell download where specific files are stored
# 1  
Old 06-06-2016
Bash to tell download where specific files are stored

The bash below will download all the files in download to /home/Desktop/folder. That works great, but within /home/Desktop/folder there are several folders bam, other, and vcf, is there a way to specify by extention in the download file where to download it to?

For example, all .pdf and .zip are downloaded to /home/Desktop/folder/other, all .bam are downloaded to /home/Desktop/folder/bam, and all .pdf.gz are downloaded to /home/Desktop/folder/vcf. Thank you Smilie.


download
Code:
file1.pdf
file2.pdf
coverage(1).zip
coverage(2).zip
001__96.bam
002_96.bam
005.vcf.gz
006.vcf.gz

Code:
while read new; do
         echo $new
aria2c -x 4 -l log.txt -c -d /home/Desktop/folder --use-head=true --http-user "<user."  --http-passwd <password> xxx://www.example.com/folder/"$new"
done < /home/download

# 2  
Old 06-06-2016
How about:

Code:
while read new; do

         case "$new" in
         *.pdf) DEST="/home/desktop/folder/other" ;;
         *.zip) DEST="/home/desktop/folder/other" ;;
         *.bam) DEST="/home/desktop/folder/bam" ;;
         *.pdf.gz) DEST="/home/desktop/folder/vcf" ;;
         *) DEST="/home/desktop/folder" ;;
         esac

        # If folder doesn't exist, make it
        [ -d "$DEST" ] || mkdir -p "$DEST"

        echo "$new => $DEST"

        aria2c -x 4 -l log.txt -c -d "$DEST" --use-head=true --http-user "<user."  --http-passwd <password> xxx://www.example.com/folder/"$new"
done < /home/download

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 06-11-2016
Thank you very much Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to move specific files to directory based on match to file

I am trying to mv each of the .vcf files in the variants folder to the folder in /home/cmccabe/f2 that the .vcf id is found in file. $2 in file will always have the id of a .vcf in the variants folder. The line in blue staring with R_2019 in file up to the -v5.6 will always be an exact match to a... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

Bash to move specific files from folders in find file

I have a directory /home/cmccabe/nfs/exportedReports that contains multiple folders in it. The find writes the name of each folder to out.txt. A new directory is then created in a new location /home/cmccabe/Desktop/NGS/API, named with the date. What I am trying to do, unsuccessfully at the moment,... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

4. Shell Programming and Scripting

Download and extract to a specific directory

Trying to download and extract a file to a specific folder, but getting an error. What am I doing wrong? Is there a way to rename the download if desired? Thank you :). curl --url https://github.com/arq5x/bedtools2/releases/download/v2.26.0/bedtools-2.26.0.tar.gz | tar -xz --output... (4 Replies)
Discussion started by: cmccabe
4 Replies

5. Shell Programming and Scripting

Csv download in a bash script

I am attempting to download a url in csv format. When I download this url in a browser excel opens up and automatically populates with comma separated values. When I try to use curl or wget I get nothing or garbage. This on the command line just hangs wget -b... (2 Replies)
Discussion started by: bash_in_my_head
2 Replies

6. Shell Programming and Scripting

Bash to download specific files and save in two folders

I am trying to download all files from a user authentication, password protected https site, with a particular extension (.bam). The files are ~20GB each and I am not sure if the below is the best way to do it. I am also not sure how to direct the downloaded files to a folder as well as external... (7 Replies)
Discussion started by: cmccabe
7 Replies

7. Shell Programming and Scripting

Bash script read specific value from files of an entire folder

Hello, I heva a problem creating a script that read specifc value from all the files of an entire folder I have a number of email files into a directory and i need to extrect from each file 2 specific values. After that i have to put them into a new file that looks like that: To: value1 ... (1 Reply)
Discussion started by: ahmenty
1 Replies

8. UNIX for Dummies Questions & Answers

[bash] Reading two files and change a specific field

Hi all. I have 2 files like these: file1 3 -2 5 4 . . . 3 3 3 4 . . . 2 2 3 4 . . . 3 -2 8 4 . . . file2 0.4242 2 3 4 . . . 2.562 7 3 4 . . . 0.7242 5 5 4 . ... (3 Replies)
Discussion started by: t_studen_t
3 Replies

9. Shell Programming and Scripting

FAQ how to download multiple specific files via command line

Hi, This evening i would like to download multiple pcap captures files in the wireshark wiki sites. My aim is to download the capture files .pcap .cap and etc on the wireshark site SampleCaptures - The Wireshark Wiki. i already used wget, lynx, htget but still their problem downloading..it seems... (1 Reply)
Discussion started by: jao_madn
1 Replies
Login or Register to Ask a Question