Visit Our UNIX and Linux User Community


Bash to add portion of text to files in directory using numerical match


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to add portion of text to files in directory using numerical match
# 1  
Old 04-02-2017
Bash to add portion of text to files in directory using numerical match

In the below bash I am trying to rename eachof the 3 text files in /home/cmccabe/Desktop/percent by matching the numerical portion of each file to lines 3,4, or 5 in /home/cmccabe/Desktop/analysis.txt. There will always be a match between the files. When a match is found each text file in /home/cmccabe/Desktop/percent is renamed adding the textafter the _ (underscore) in /home/cmccabe/Desktop/analysis.txt to each line.

text file in /home/cmccabe/Desktop/percent - there could be a maximum of 3 files in this directory
Code:
00-0000_fbn1_20xcoverage.txt
01-0101_fbn1_20xcoverage.txt
02-0202_fbn1_20xcoverage.txt

text file in /home/cmccabe/QC/analysis.txt
Code:
  status: complete 
id names:  
00-0000_Last-First 
01-0101_LastN-FirstN
02-0202_La-Fi

desired result in /home/cmccabe/QC/percent
Code:
  00-0000_Last-First_fbn1_20xcoverage.txt
01-0101_LastN-FirstN_fbn1_20xcoverage.txt
02-0202_La-Fi_fbn1_20xcoverage.txt

The set -x shows that the files are renamed correctly but they are not moved to /home/cmccabe/Desktop/percent. Currently the 3 text files in there are the same as above (without the name added to them).

set -x
Code:
+ IFS=_
+ read -r id newname
++ tail -n+3 /home/cmccabe/QC/analysis
++ find . -name '00-0000*.txt' -printf %f
+ oldfilename=00-0000_fbn1.txt
+ '[' -n 00-0000_fbn1.txt ']'
+ echo mv '"00-0000_fbn1.txt"' '"00-0000_Last-First_fbn1.txt"'
mv "00-0000_fbn1.txt" "00-0000_Last-First_fbn1.txt"
+ IFS=_
+ read -r id newname
++ find . -name '01-0101*.txt' -printf %f
+ oldfilename=01-0101_fbn1.txt
+ '[' -n 01-0101_fbn1.txt ']'
+ echo mv '"01-0101_fbn1.txt"' '"01-0101_LastN-FirstN_fbn1.txt"'
mv "01-0101_fbn1.txt" "01-0101_LastN-FirstN_fbn1.txt"
+ IFS=_
+ read -r id newname
++ find . -name '02-0202*.txt' -printf %f
+ oldfilename=02-0202_fbn1.txt
+ '[' -n 02-0202_fbn1.txt ']'
+ echo mv '"02-0202_fbn1.txt"' '"02-0202_La-Fi_fbn1.txt"'
mv "02-0202_fbn1.txt" "02-0202_La-Fi_fbn1.txt"
+ IFS=_
+ read -r id newname

Description of what I think is going on (I will try to include these:
Code:
read the analysis.txt file split each line (i.e 00-0000_Last-First) to two fields using _ as delimiter:
id=00-000
newname=Last-First
  Then using this file id read from file analysis.txt check (using find) to see if a file exists starting with the same id.
The matching filename is returned in variable $oldfilename.
If this variable is not empty mv the files.
tail -n+3 is used to ignore the first two lines of the analysis.txt

bash
Code:
while IFS="_" read -r id newname;do
#echo "id=$newid - newname=$newname"  #for cross check 
oldfilename=$(find . -name "${id}*.txt" -printf %f)
[ -n "$oldfilename" ] && echo mv \"$oldfilename\" \"${id}_${newname}_${oldfilename#*_}\";
done < <(tail -n+3 /home/cmccabe/QC/analysis.txt)

I am not sure why the mv is working and the 3 text files remain the same. The echo shows them renamed correctly. Thank you Smilie.

Last edited by cmccabe; 04-02-2017 at 11:33 AM.. Reason: fixed format
# 2  
Old 04-02-2017
Take a look at your code again. There is no mv command in your code; only an echo command. If the echo shows that your script is correctly determining what arguments should be passed to mv, what happens if you change the line:
Code:
[ -n "$oldfilename" ] && echo mv \"$oldfilename\" \"${id}_${newname}_${oldfilename#*_}\";

in your code to:
Code:
[ -n "$oldfilename" ] && mv "$oldfilename" "${id}_${newname}_${oldfilename#*_}";

(i.e., remove the echo and the backslashes)?
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 04-02-2017
That fixed it, but now I have the same problem I did before. The bash, as is works great as long as all 3 files are in the directory /home/cmccabe/QC/percent.

Since I run a process on each file, after it gets renamed it is moved to the directory. When the next filename gets renamed, the other one gets duplicated. If there is a third the first filename repeats 3 times the second filename duplicated and third is fine.

So if in /home/cmccabe/QC/percent

Code:
00-0000_Last-First_fbn1.txt

code executes a second time:
Code:
00-0000_Last-First_Last-First_fbn1.txt  ---- file already in dir
01-0101_LastN-FirstN_fbn1.txt

code executes a third time:
Code:
00-0000_Last-First_Last-First_Last-First_fbn1.txt  ---- file already in dir
01-0101_LastN-FirstN_LastN-FirstNfbn1.txt   --- this is the 2nd file in dir
02-0202_La-Fi_fbn1.txt

Maybe I just need to only execute the command once after all 3 files are in the directory? If a file is in the directory already it has been renamed already but I'm not sure how to prevent the duplication. Thank you Smilie.

Previous Thread | Next Thread
Test Your Knowledge in Computers #78
Difficulty: Easy
Linux was inspired by MINIX
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to rename portion of file using match to another

In the portion of bash below I am using rename to match the $id variable to $file and when a match (there will alwsys be one) is found then the $id is removed from each bam and bam.bai in $file and _test is added to thee file name before the extension. Each of the variables is set correctly but... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Bash to move specific files to directory based on match to file

I am trying to mv each of the .vcf files in the variants folder to the folder in /home/cmccabe/f2 that the .vcf id is found in file. $2 in file will always have the id of a .vcf in the variants folder. The line in blue staring with R_2019 in file up to the -v5.6 will always be an exact match to a... (4 Replies)
Discussion started by: cmccabe
4 Replies

3. Shell Programming and Scripting

Match and store numerical prefix to update files

In the bash below the unique headers of each vcf.gz are stored in a text file with the same name. That is if 16-0000-file.vcf.gz was used the header text file would be 16-0000-file_header.txt. There can be multiple vcf.gz in a directory, usually 3, that I need to fix the header in each file before... (6 Replies)
Discussion started by: cmccabe
6 Replies

4. Shell Programming and Scripting

Display match or no match and write a text file to a directory

The below bash connects to a site, downloads a file, searches that file based of user input - could be multiple (all that seems to work). What I am not able to figure out is how to display on the screen match found or no match found" and write a file to a directory (C:\Users\cmccabe\Desktop\wget)... (4 Replies)
Discussion started by: cmccabe
4 Replies

5. Shell Programming and Scripting

Text match in two files

Trying to match the text from file1 to file2 and print what matches in a new file (match.txt) and what does not in another (missing.txt). awk -F'|' 'NR==FNR{c++;next};c > 0' flugent.txt IDT.txt > match.txt Thank you :). (8 Replies)
Discussion started by: cmccabe
8 Replies

6. Shell Programming and Scripting

Perl: batch replace a portion of text in files

Hi all, What I would like to achieve is to batch change the code below in every pdf in a given directory (each pdf is uncompressed so that can be easily edited). An example of the javascript code: if (this.hostContainer) { try { this.hostContainer.postMessage(pp_216', 15259]);... (3 Replies)
Discussion started by: menteith
3 Replies

7. Shell Programming and Scripting

search a regular expression and match in two (or more files) using bash

Dear all, I have a specific problem that I don't quite understand how to solve. I have two files, both of the same format: XXXXXX_FIND1 bla bla bla bla bla bla bla bla bla bla bla bla ======== (return) XXXXXX_FIND2 bla bla bla bla bla bla (10 Replies)
Discussion started by: TheTransporter
10 Replies

8. Programming

Fuzzy Match Logic for Numerical Values

I have searched the internet (including these forums) and perhaps I'm not using the right wording. What I'm looking for is a function (preferably C) that analyzes the similitude of two numerical or near-numerical values, and returns either a true/false (match/nomatch) or a return code that... (4 Replies)
Discussion started by: marcus121
4 Replies

9. Shell Programming and Scripting

Counting files in a directory that match a pattern

I have 20 files in a direcotry like BARE01_DLY_MKT_YYYYMMDD. The MKT differes for all these files but the remaining syntax remains the same for a particular day. If I am checking for today I need to make sure that there are 20 files that start with BARE01_DLY_MKT_20060720. How can I write a... (31 Replies)
Discussion started by: dsravan
31 Replies

10. Shell Programming and Scripting

Match portion of the filename

hi, I have a script which accept filename and process them, however, one of the file needs 'special' handling so I need to identify it, say the filename contains the word "STOCK" (i.e. NEWYORKSTOCKLIST20060627.txt), I want to check if the filename contains the word "STOCK", how can I do that?... (1 Reply)
Discussion started by: mpang_
1 Replies

Featured Tech Videos