Sponsored Content
Top Forums Shell Programming and Scripting Bash to trim folder and files within a path that share a common file extension Post 303032421 by Don Cragun on Sunday 17th of March 2019 08:45:42 AM
Old 03-17-2019
Quote:
Originally Posted by cmccabe
The bash will trim the folder to trim folder. Within each of the folders (there may be more than 1) and the format is always the same, are several .bam and matching .bam.bai files (file structure) and the bashunder that executes and trims the .bam as expected but repeats the.bam.bai extentions after trimming those files. xxxx_0113_xxx_xxx.bam.bai.bam.bai also in the set -xv. I think the .bam extension common to both may be causing the repeat but am not sure. Removing the .bam.bai from the mv did not fix the repeats. The end goal is to trim the folders and the files within each of the folders and I am not sure if the nested loops are the best way (probably not). Thank you Smilie.


bash to trim folder
Code:
for folder in /home/cmccabe/rename/*/ ; do  ## start loop in subdirectory
     mv  "$folder"  "${folder%%-v5.6*}"  ## trim folder name
done  ## close loop

folder in /home/cmccabe/rename
Code:
R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions

trim folder
Code:
R_2019_01_30_14_24_53_user_S5-0271-95

file structure in each /home/cmccabe/rename/<folder>
Code:
xxxx_0111_xxx_xxx_R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions.bam
xxxx_0111_xxx_xxx_R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions.bam.bai
xxxx_0113_xxx_xxx_R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions.bam
xxxx_0113_xxx_xxx_R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions.bam.bai

bash to trim files
Code:
for d in /home/cmccabe/rename/* ; do  ## start loop in parentdir
  if [ -d "$d" ]; then  ##  grab subdir and store in parentdir/subdir in $d
     subdir="$(basename $d)"  ## define sub-directory
  fi  ## end if
 for bam in "${d}"/*.bam ; do ## iterate through each file in parentdir and read into bam
   for bai in "${d}"/*.bam.bai ; do ## iterate through each file in parentdir/subdir and read into bai
     bam_path_removed=$(echo $bam| awk -F/ '{print $NF}') ## cut text before last /
     bai_path_removed=$(echo $bai| awk -F/ '{print $NF}') ## cut text before last /
     bam_trim=$(echo "$bam_path_removed"|cut -f1,2,3,4 -d'_')
     bai_trim=$(echo "$bai_path_removed"|cut -f1,2,3,4 -d'_')
       mv "${bam}" "${d}/${bam_trim}".bam ## rename all bam
       mv "${bai}" "${d}/${bai_trim}".bam.bai ## rename all bai
  done ## close loop
 done ## close loop
done ## close loop

desired trim files
Code:
xxxx_0111_xxx_xxx.bam
xxxx_0111_xxx_xxx.bam.bai
xxxx_0113_xxx_xxx.bam
xxxx_0113_xxx_xxx.bam.bai

set -xv w/ echo mv
Code:
+ echo mv /home/cmccabe/rename/R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions/xxxx_0111_xxx_xxx_R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions.bam.bai /home/cmccabe/rename/R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions/xxxx_0111_xxx_xxx.bam.bai

set -xv w/o echo mv
Code:
mv /home/cmccabe/rename/R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions/xxxx_0111_xxx_xxx.bam.bai /home/cmccabe/rename/R_2019_01_30_14_24_53_user_S5-0271-95-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions/xxxx_0111_xxx_xxx.bam.bai.bam.bai

What you have shown us above makes it look like you may have moved a bunch of your *.bam.bai files into *.bam.bai.bam.bai files and possibly moved *.bam files into *.bam.bam files. The purpose of the echo commands was to make sure that the mv commands that would be executed looked good before actually moving files. The fact that the echo set -xv output did not show the same filenames as the mv set -xv output seems to imply that when the echo was removed from the echo mv lines in your script, something else in your script was changed than just removing the echo in front of the two mv commands.

Since the output from the set -xv trace showed that it was going to rename files in the wrong directory, why did you remove the echo and run it again? The purpose of having the echo in there is so that you can verify that the command being echoed is the command that you want the script to actually perform when run a second time with the echos removed.

Please show us the output from the command:
Code:
find /home/cmccabe/rename/ \( -type d -o -name '*.bam*' \) -exec ls -ld {} +

so we can see how things stand now. Please also tell us what operating system you're using. (PLEASE always tell us what operating system and shell you're using when you start a new thread.)

I don't know if you have tried running the code Peasant suggested in post #2 in this thread. I'm afraid the code Peasant suggested might only work with the file hierarchy you described before any files were moved. (Note that I haven't tried to figure out what his code will do if it starts with your modified file hierarchy instead of what may be the current file hierarchy.) Do you have backups so you can restore that original state? If not, I'm hoping that with the output from the find command above we'll be able to find a way to get to where you want to be without losing any data.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. AIX

trim file name extension????

Dear Chaps, What will I do if, I am not sure about the length of the file name, but only one thing that I want to remove only the last extension. e.g. abcdXXXXXX.pqrXXXXX.asc (X is any character) I want to trim only .asc (or,watever) so that resultant file name would be like... (1 Reply)
Discussion started by: vishal_ranjan
1 Replies

2. Windows & DOS: Issues & Discussions

How can I upload a zip folder on a unix path from my windows folder?

Hello, I am an amature at UNIX commands and functionality. Please could you all assist me by replying to my below mentioned querry : How can I upload a zip folder on a unix path from my windows folder? Thanks guys Cheers (2 Replies)
Discussion started by: ajit.yadav83
2 Replies

3. Shell Programming and Scripting

command to list files with path & date in a folder

Hi, I need command to display files with full path and date of files where are generated at every 5hrs in a folder. eg: /u01/app/test/orjthsd_1_1 Sun May 10 19:03:26 2009 /u01/app/test/weoiusd_1_1 Sun May 10 21:00:26 2009 thanks saha (3 Replies)
Discussion started by: saha
3 Replies

4. Shell Programming and Scripting

move set of files to the target path with different extension

I have the following files in the dir /home/krishna/datatemp abc.xml cde.xml asfd.txt asdf_20120101-1.xml asdf_20120101-2.xml asdf_20120101-3.xml asdf_20120101-4.xml Now I need to move the files having the pattern asdf_20120101-*.xml to the dir /home/krishna/dataout with the extn as... (1 Reply)
Discussion started by: kmanivan82
1 Replies

5. Shell Programming and Scripting

Moving files from parent path to multiple child path using bash in efficient way

Hi All, Can you please provide some pointers to move files from Base path to multiple paths in efficient way.Folder Structure is already created. /Path/AdminUser/User1/1111/Reports/aaa.txt to /Path/User1/1111/Reports/aaa.txt /Path/AdminUser/User1/2222/Reports/bbb.txt to... (6 Replies)
Discussion started by: karthikgv417
6 Replies

6. Shell Programming and Scripting

Rename specific file extension in directory with match to another file in bash

I have a specific set (all ending with .bam) of downloaded files in a directory /home/cmccabe/Desktop/NGS/API/2-15-2016. What I am trying to do is use a match to $2 in name to rename the downloaded files. To make things a more involved the date of the folder is unique and in the header of name... (1 Reply)
Discussion started by: cmccabe
1 Replies

7. Linux

How to print full path name along with file extension?

Hi I have a requirement like this: /abc/a/x.txt /abc/a/y.txt /abc/b/x.gz /abc/b/y.txt I need output like this: /abc/a:*.txt /abc/b:*.txt /abc/b:*.gz I have tried find /abc -type f -name "*.*" ||awk -F . '{print $NF}' it is print only extensions without path name. Please... (5 Replies)
Discussion started by: lijjumathew
5 Replies

8. Shell Programming and Scripting

Find common lines with one file and with all of the files in another folder

Hi! I would like to comm -12 with one file and with all of the files in another folder that has a 100 files or more (that file is not in that folder) to find common text lines. I would like to have each case that they have common lines to be written to a different output file and the names of the... (6 Replies)
Discussion started by: Eve
6 Replies

9. Shell Programming and Scripting

Find common lines between all of the files in one folder

Could it be possible to find common lines between all of the files in one folder? Just like comm -12 . So all of the files two at a time. I would like all of the outcomes to be written to a different files, and the file names could be simply numbers - 1 , 2 , 3 etc. All of the file names contain... (19 Replies)
Discussion started by: Eve
19 Replies

10. Shell Programming and Scripting

Bash to create sub directories from specific file extension

In the bash below I am trying to create sub-directories inside a directory from files with specific .bam extensions. There may be more then one $RDIR ing the directory and the .bam file(s) are trimmed (removing the extension and IonCode_0000_) and the result is the folder name that is saved in... (2 Replies)
Discussion started by: cmccabe
2 Replies
All times are GMT -4. The time now is 07:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy