Bash to move specific files from folders in find file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to move specific files from folders in find file
# 1  
Old 09-16-2016
Bash to move specific files from folders in find file

I have a directory /home/cmccabe/nfs/exportedReports that contains multiple folders in it. The find writes the name of each folder to out.txt. A new directory is then created in a new location /home/cmccabe/Desktop/NGS/API, named with the date. What I am trying to do, unsuccessfully at the moment, is copy each specific file type .bam from each folder in out.txt to the new date folder in /home/cmccabe/Desktop/NGS/API/$DATE.

For example, after the find command out.txt:
Code:
Folder1
Folder2

within each of those folders there are 3 .bam files to move to the new directory/home/cmccabe/Desktop/NGS/API/$DATE. Thank you Smilie.

Code:
#!/bin/bash

cd /home/cmccabe/nfs/exportedReports
find . -maxdepth 1 -type d | cut -c 3- | awk -F _ 'NF > 1' > out.txt

DATE=`date +%-m-%-d-%Y` 
        mkdir -p /home/cmccabe/Desktop/s5/$DATE

MY_DIR="/home/cmccabe/nfs/exportedReports/out.txt"
DEST="/home/cmccabe/Desktop/NGS/API/$DATE"
FILEEXT=".bam"

NEWEST=`ls -tr1d "${MY_DIR}/"*.${FILEEXT} 2>/dev/null | tail -1`

if [ -z "${NEWEST}" ] ; then
    echo "No file to copy"
    exit 1
elif [ -d "${NEWEST}" ] ; then
    echo "The most recent entry is a directory"
    exit 1
else
    echo "Copying ${NEWEST}"
    cp -p "${NEWEST}" "${DEST}"
fi

# 2  
Old 09-16-2016
Hi,

Code:
NEWEST=`ls -tr1d "${MY_DIR}/"*.${FILEEXT} 2>/dev/null | tail -1`

Above code is not doing what you explained in your post.

You need to read line by line in out.txt file then find / list the files with *.bam extension , copy etc.

It may be done in oneliner , if you can add more description about structure of Folder1,Folder2 and /home/cmccabe/nfs/exportedReports with clear example.

if not, try like this:
Code:
cat out.txt
DIR1
DIR2
$ ls -a DIR1/
.  ..  123.bam  abc.bam  xyz.log
$ ls -a DIR2/
.  ..  cool.txt  me.bam

Sample code written here to update your script.

Code:
a=/home/cmccabe/nfs/exportedReports;DEST="/home/cmccabe/Desktop/NGS/API/$DATE";
while read line; do find $a/$line -name "*.bam" -exec cp {} $DEST \;
done < out.txt 
$ ls -a $DEST
.  ..  123.bam  abc.bam  me.bam


Last edited by greet_sed; 09-16-2016 at 01:56 PM.. Reason: Add sample code
This User Gave Thanks to greet_sed For This Post:
# 3  
Old 09-16-2016
The below is all most working, there are sub-directories in each folder and the find and cp doesn't need to search these, as the .bam files are all in the main folder. Thank you Smilie.

I am getting an error:
Code:
cp: cannot stat ‘/home/cmccabe/nfs/exportedReports/Auto_user_S5-00580-5-Medexome_66_030/plugin_out/coverageAnalysis_out.39/IonXpress_007/scraper/link.summary.bam': No such file or directory

--- this is a subdirectory that can be excluded from the find, but I am not sure how.

Code:
#!/bin/bash

cd /home/cmccabe/nfs/exportedReports
find . -maxdepth 1 -type d | cut -c 3- | awk -F _ 'NF > 1' > out.txt

DATE=`date +%-m-%-d-%Y` 
        mkdir -p /home/cmccabe/Desktop/s5/$DATE

a=/home/cmccabe/nfs/exportedReports;DEST="/home/cmccabe/Desktop/s5/$DATE";
while read line; do find $a/$line -name "*.bam" -exec cp {} $DEST \;
done < out.txt

# 4  
Old 09-16-2016
Hello cmccabe,

If I understood your requirement correctly(though I am not sure) it is to copy a specific extension files from source folder to target folder, so could you please try following and let me know if this helps you(not tested though).
Code:
find . -maxdepth 1 -name "test*" -exec echo "cp -t /tmp" {} \+

So in above code you could put .*bamfiles in place of test*and could put location in place of /tmpabove(destination dire I have put echo(in bold and underline) for testing purposes, so if you are happy with above command's result you could remove that echo. You could set the value of maxdepthas per your requirements too.
Please do let us know how it goes then.

Thanks,
R. Singh

Last edited by RavinderSingh13; 09-16-2016 at 02:50 PM..
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 09-16-2016
I will try to set the maxdepth. Thank you Smilie.
# 6  
Old 09-16-2016
Is the goal to copy the *.bam files in *_* folders, i.e. *_*/*.bam files?
Then try this simple one
Code:
DATE=`date +%-m-%-d-%Y` 
FROM="/home/cmccabe/nfs/exportedReports"
DEST="/home/cmccabe/Desktop/s5/$DATE"
if
  cd "$FROM" &&
  mkdir -p "$DEST"
then
  cp *_*/*.bam "$DEST"
fi

This User Gave Thanks to MadeInGermany For This Post:
# 7  
Old 09-16-2016
Quote:
Originally Posted by cmccabe
I will try to set the maxdepth. Thank you Smilie.
Hello cmccabe,

In your post#3 I saw you asked how to exclude any directory or sub-directory in findcommand then following could help you in same. To exclude a directory and only print all the file names following may help you in same.
Code:
find . -maxdepth 2 -type d -name "again_test*" -prune -o -type f -name "test*" -print

If you are happy with above command then you could copy files to the destination directory by following command.
Code:
find . -maxdepth 2 -type d -name "again_test*" -prune -o -type f -name "test*" -exec cp -t /tmp {} \+

So in above codes change values of again_test*to that directory which you want to exclude, test*with those file extensions/names which you want to find here,/tmpwith your destination path please. Change maxdepthvalue as per your need here. Kindly do test the same and please do let us know how it goes for this.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to move specific files to directory based on match to file

I am trying to mv each of the .vcf files in the variants folder to the folder in /home/cmccabe/f2 that the .vcf id is found in file. $2 in file will always have the id of a .vcf in the variants folder. The line in blue staring with R_2019 in file up to the -v5.6 will always be an exact match to a... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. UNIX for Advanced & Expert Users

Find files in specific folders

Hi Team, I am new to the linux commands and I really need help . I would be really thankful if I can get some inputs. I have below folders in the path "/home/temp" 20170428 20170427 20170429 changes tempI need to get the files generated in the last 15 mins in all the above folders... (4 Replies)
Discussion started by: JackJinu
4 Replies

3. Shell Programming and Scripting

How can i move folders and its content if folder is older than 1,5 days and keep subdirs in bash?

Hello all, do you know any way i can i move folders and its content if folder is older than 1,5 days in bash? I tried: find /home/xyz/DATA/* -type d -ctime +1.5 -exec mv "{}" /home/xyz/move_data_here/ \;All i got was that Files from DATA /home/xyz/DATA/* ended messed up in... (1 Reply)
Discussion started by: ZerO13
1 Replies

4. Shell Programming and Scripting

Move multiple files 4rm Source to different target folders based on a series num in the file content

Dear Experts my scenario is as follows... I have one source folder "Source" and 2 target folders "Target_123456" & "Target_789101". I have 2 series of files. 123456 series and 789101 series. Each series has got 3 types of fiels "Debit", "Refund", "Claims". All files are getting... (17 Replies)
Discussion started by: phani333
17 Replies

5. Shell Programming and Scripting

Move specific folders and subfolders in a directory

I am trying to move specific folders and subfolders within a directory using the below. I can see the folders to move and they are at the location, but I am getting an error. Thank you :). mv -v /home/cmccabe/Desktop/NGS/API/6-10-2016{bam/{validation,coverage},bedtools /media/cmccabe/"My... (6 Replies)
Discussion started by: cmccabe
6 Replies

6. Shell Programming and Scripting

Bash to download specific files and save in two folders

I am trying to download all files from a user authentication, password protected https site, with a particular extension (.bam). The files are ~20GB each and I am not sure if the below is the best way to do it. I am also not sure how to direct the downloaded files to a folder as well as external... (7 Replies)
Discussion started by: cmccabe
7 Replies

7. Shell Programming and Scripting

How to find a word and move it a specific location in xml file using perl?

Hi friends, I have one XML file having below structure :- INput XML file :- <?xml version="1.0" encoding="UTF-8"?> <START> <A=value1> <attr name1="a1"> </A> <B=value2> <attr name2="b1"> <attr name3="c1"> </B> </START> output xml file should be === (3 Replies)
Discussion started by: harpal singh
3 Replies

8. Shell Programming and Scripting

Find file that matches today's date in filename and move to /tmp in bash

I'm having problems with my bash script. I would like to find a file matching today's date in the filename, i.e. my_file_20120902.txt and then move it to a different directory, i.e. /tmp. Thanks. (1 Reply)
Discussion started by: jamesi
1 Replies

9. Shell Programming and Scripting

Move files to Folders

Hi Friends, Below is my requirement and i am not clear how to approach this issue in unix programming. I have a folder with 2500 files. The files are in below format. 1234_name1.txt 1234_name123.txt 4567_name1.txt 4567_name123.txt and i need a program which will read each file from this... (5 Replies)
Discussion started by: diva_thilak
5 Replies

10. UNIX for Dummies Questions & Answers

Move folders containing certain files

Hello, How can I move just the folders that contains files modified n days ago? Source tree: |-- SourceFolder | |-- Subfolder1 | | |-- file1.dat | | `-- file2.dat | |-- Subfolder2 | | |-- filea.dat | | `-- fileb.dat Destination tree: |-- ... (3 Replies)
Discussion started by: xavix
3 Replies
Login or Register to Ask a Question