Bash to rename file after second occurence of underscore


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to rename file after second occurence of underscore
# 1  
Old 11-23-2016
Bash to rename file after second occurence of underscore

I am trying to use bash to remove the text in all filenames after the second _ in specific files that end in .bam or .vcf. However the bash errors looking for the files in the directory. Thank you Smilie.

files in directory
Code:
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

desired output
Code:
IonXpress_007.bam
IonXpress_008.bam
IonXpress_009.bam
IonXpress_007.vcf
IonXpress_008.vcf
IonXpress_009.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

Bash
Code:
for file in /home/cmccabe/Desktop/NGS/test/$filename/{*.bam *.vcf}; do
   mv -- "$file" "${file%%_*_*}.bam .vcf"
done


Last edited by cmccabe; 11-23-2016 at 08:38 PM.. Reason: added details
# 2  
Old 11-23-2016
Maybe this one:

Code:
file=IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
echo ${file//${file#*_[0-9][0-9][0-9]}}.bam

#Output 
IonXpress_008.bam

Have fun with figuring out, how it's working. Smilie

This one is a bit more flexible:

Code:
file=IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
file2="${file//${file#*_*_}}"
echo "${file2:0: -1}.bam"

#Output 
IonXpress_008.bam


Last edited by stomp; 11-24-2016 at 06:43 AM..
This User Gave Thanks to stomp For This Post:
# 3  
Old 11-24-2016
Quote:
Originally Posted by cmccabe
I am trying to use bash to remove the text in all filenames after the second _ in specific files that end in .bam or .vcf. However the bash errors looking for the files in the directory. Thank you Smilie.

files in directory
Code:
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

desired output
Code:
IonXpress_007.bam
IonXpress_008.bam
IonXpress_009.bam
IonXpress_007.vcf
IonXpress_008.vcf
IonXpress_009.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

Bash
Code:
for file in /home/cmccabe/Desktop/NGS/test/$filename/{*.bam *.vcf}; do
   mv -- "$file" "${file%%_*_*}.bam .vcf"
done

Please try the following and if you like the result modify the highlighted echo to be just the mv command
Code:
for f in /home/cmccabe/Desktop/NGS/test/${filename}/*.{bam,vcf}; do
 if [[ -e $f ]]; then
   p="${f%/*}"
   i="${f#$p/}"
   e="${f##*.}"
   ci="${i%%_[A-Z]*}"
   echo mv -v "$f" "${p}/${ci}.${e}"
 fi
done

This User Gave Thanks to Aia For This Post:
# 4  
Old 11-24-2016
Hi.

I like the rename command:
Code:
rename perl-expression files

Here's how it could work:
Code:
#!/usr/bin/env bash

# @(#) s2       Demonstrate group rename with perl regular expression, rename.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C rename pass-fail dixf
dixf mv rename pass-fail dixf

FILE=${1-data1}
rm Ion*
E=expected-output.txt

pl " Input data file $FILE, creating files:"
touch $( cat $FILE )
ls Ion*

pl " Expected output:"
cat $E

pl " Results:"
# Pattern and sample match; ".*?" is a non-greedy match.
#
# IonXpress_007
# ^(.*?_.*?)
#              _MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome
#              _.*
#                                                                    .bam
#                                                                   [.]

rename 's/^(.*?_.*?)_.*[.]/$1./' *bam *vcf
( ls -1 Ion*bam
ls -1 Ion*vcf
ls -1 Ion*txt ) |
tee f1

pl " Verify results if possible:"
C=$HOME/bin/pass-fail
[ -f $C ] && $C || ( pe; pe " Results cannot be verified." ) >&2

exit 0

producing:
Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie) 
bash GNU bash 4.3.30
rename /usr/bin/rename using File::Rename version 0.20
pass-fail (local) 1.9
dixf (local) 1.19

mv      move (rename) files (man)
Path    : /bin/mv
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYSV ...)
Help    : probably available with --help

rename  renames multiple files (man)
Path    : /usr/bin/rename
Type    : symbolic link to /etc/alternatives/rename ...)
Modules : (for perl codes)
 strict 1.08
 File::Rename   0.20
 Pod::Usage     1.63

pass-fail       Compare files, issue pass or fail verdict, <f1>, <expected-output.txt>. (what)
Path    : ~/bin/pass-fail
Length  : 90 lines
Type    : a /usr/bin/env bash script, ASCII text executable
Shebang : #!/usr/bin/env bash

dixf    Display information about executable file, script or binary. (what)
Path    : ~/bin/dixf
Length  : 153 lines
Type    : a /usr/bin/env bash script, ASCII text executable
Shebang : #!/usr/bin/env bash
Help    : probably available with --help,--man,--usage

-----
 Input data file data1, creating files:
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf

-----
 Expected output:
IonXpress_007.bam
IonXpress_008.bam
IonXpress_009.bam
IonXpress_007.vcf
IonXpress_008.vcf
IonXpress_009.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

-----
 Results:
IonXpress_007.bam
IonXpress_008.bam
IonXpress_009.bam
IonXpress_007.vcf
IonXpress_008.vcf
IonXpress_009.vcf
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.txt

-----
 Verify results if possible:

-----
 Comparison of 9 created lines with 9 lines of desired results:
 Succeeded -- files (computed) f1 and (standard) expected-output.txt have same content.

Different repositories may have different versions of rename.

Best wishes ... cheers, drl

Last edited by drl; 11-24-2016 at 04:24 PM.. Reason: Edit 1: typo in perl RE comment
This User Gave Thanks to drl For This Post:
# 5  
Old 11-24-2016
Thank you all very much and enjoy your holiday Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to rename portion of file using match to another

In the portion of bash below I am using rename to match the $id variable to $file and when a match (there will alwsys be one) is found then the $id is removed from each bam and bam.bai in $file and _test is added to thee file name before the extension. Each of the variables is set correctly but... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Replace a space with underscore in a file

i have a file with following data. { EqName "Tan 1" .... .... } { EqName "Sin 2" ... ... } I have to replace the value of EqName to Tan_1 and Sin_2 in file.Can i use sed or awk ? cat file|grep EqName|awk '{print $2 $3}'|sed -i 's//_/g' I tried with this but it... (2 Replies)
Discussion started by: Jag02
2 Replies

3. Shell Programming and Scripting

Bash to copy file 3 times and rename based on another file

In the below bash I am trying to copy the only text file (always only one) in /home/cmccabe/Desktop/list/QC/metrics.txt and rename each of the 3 text files according to /home/cmccabe/Desktop/test/list.txt using lines 3, 4 ,5. This format (that is list.txt) is always 5 lines. Thank you :). ... (12 Replies)
Discussion started by: cmccabe
12 Replies

4. Shell Programming and Scripting

Rename specific file extension in directory with match to another file in bash

I have a specific set (all ending with .bam) of downloaded files in a directory /home/cmccabe/Desktop/NGS/API/2-15-2016. What I am trying to do is use a match to $2 in name to rename the downloaded files. To make things a more involved the date of the folder is unique and in the header of name... (1 Reply)
Discussion started by: cmccabe
1 Replies

5. Shell Programming and Scripting

BASH function to rename file to last mondays date

I'm trying to create a bash function that will allow me to rename a file name emails.txt to last monday's date renem () { d=`date -d last-monday` mv ~/emails.txt ~/myemails/$d } but i en up wit this error any suggestion mv: target `2014' is not a directory (3 Replies)
Discussion started by: robshal
3 Replies

6. Shell Programming and Scripting

How to rename all files and folder containing underscore?

I want to rename all files and folder containing underscore in name and replace underscore with hyphen. Currently I am using following code, rename '_' '-' */*/* It was working but now it is showing me "Argument list too long" Please help! (2 Replies)
Discussion started by: opticalpigion
2 Replies

7. Shell Programming and Scripting

HOW TO - Bash REGEX - Print Error when More then One Occurence is Found Consecutively?

Hello All, Bash Version: 4.1.10(1) I'm trying to "verify" some user input. The User input will contain the "Absolute Path" a "Command" and any "Options" of that Command. For Example, say the user's input is: user_input="/usr//local/myExample/check_process -p 'java' -w 10 -c 20" I... (6 Replies)
Discussion started by: mrm5102
6 Replies

8. Shell Programming and Scripting

Need bash script to ping the servers and rename the output file each time the script is ran

HI, I have a file serverlist in that all host names are placed. i have written a small script #./testping #! /bin/bash for i in `cat serverlist` do ping $i >> output.txt done so now it creates a file output.txt till here fine.. now each time i run this script the output file... (4 Replies)
Discussion started by: madhudeva
4 Replies

9. Shell Programming and Scripting

sorting file names with hyphen and underscore

Hi, I have two types of file names filename1_12345 or filename1-12345 at the same time I have second type filename2-12345 in a txt file. I am trying to write awk script that will sort these names with below pattern ALL file names like filename1_12345 and filename1-12345 will go to... (1 Reply)
Discussion started by: rider29
1 Replies

10. Shell Programming and Scripting

bash script to rename in mass

Basically, I have a huge amount of files (ripped audiobooks) that all have the same garbage in their filenames. I'm wondering how to go about writing a bash script to mass rename them. Example filenames as they stand now: The First CD - 1x01 - Title 1.mp3 The First CD - 1x02 - Title 2.mp3... (4 Replies)
Discussion started by: audiophile
4 Replies
Login or Register to Ask a Question