Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Bash to match and store line as variable

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to match and store line as variable
# 1  
Old 01-17-2017
Bash to match and store line as variable
The bash below loops through a specific directory dir and finds and writes the oldest folder to a variable called $filename.

Code:
#!/bin/bash

# oldest folder stored as variable for analysis, version log created, and quality indicators matched to run
dir=/home/cmccabe/Desktop/NGS/test

find "$dir" -maxdepth 1 -mindepth 1 -type d -printf '%T+\t%P\0' | sort -rz |
while read -r -d $'\t' time && read -r -d '' filename
do
    printf "The oldest folder is $filename, created on $time and analysis done using v1.4 by $USER at $(date "+%D %r")\n" >> /home/cmccabe/medex.logs/folder.log
    awk -v FL="$filename" '
         FNR == 1 {filenum++}
         filenum==1 && index($0, FL) { 
              match($0, "_0*([0-9]+)/")
              FNUM=substr($0,RSTART+1,RLENGTH-2)
              gsub(/^0+/,"", FNUM)
          }
          filenum==2 && $0 ~ FNUM".pdf$"' /home/cmccabe/s5_files/downloads/list /home/cmccabe/s5_files/pdf/pdf > /home/cmccabe/s5_files/pdf/output
    break
done

Result output stored in /home/cmccabe/medex.logs/folder.log
Code:
The oldest folder is R_2017_01_13_12_11_56_user_S5-00580-24-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 12:41:03 PM

The variable (in bold) is also written to a log file stored at/home/cmccabe/medex.logs/folder.log.

So the R_2017_01_13_12_11_56_user_S5-00580-24-Medexome would be extracted and saved as $filename for the below processes to use.
Code:
 (these are just two of 50 that all use the $filename variable)

mkdir -p /home/cmccabe/Desktop/NGS/API/$filename/{bam/{validation,coverage},bedtools,fastqc,hgmd,igv,panel/{reads,},panel/20x/{base,gene,percent},panel/{reads,},panel/30x/{base,gene,percent},pdf,picard,torrent,vcf/overall/{annovar,stats},vcf/panel/{annovar,}}

# create analysis log
echo "analysis done using v1.4 by $USER at $(date "+%D %r")" > /home/cmccabe/Desktop/NGS/API/$filename/analysis.txt

So basically store the portion in bold from the log in the filename variable so the other lines can use it to execute.
The problem is there could be multiple lines in the log, but only one returned from the bash. The line that is returned from the bash
needs to be matched to a line in log and stored as filename.

So in the log there may be 3 lines:
Code:
R_2017_01_13_12_11_56_user_S5-00580-24-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 12:41:03 PM
R_2017_01_13_12_11_56_user_S5-00580-25-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 1:22:09 PM
R_2017_01_13_12_11_56_user_S5-00580-26-Medexome, created on 2017-01-17+11:31:02.5035483130 and analysis done using v1.4 by cmccabe at 01/17/17 2:11:44 PM

But on 1 returned by the bash
Code:
R_2017_01_13_12_11_56_user_S5-00580-25-Medexome

So, R_2017_01_13_12_11_56_user_S5-00580-25-Medexome would be stored in the $filename variable.

I am really not sure how to do this and hope I posted enough information. Thank you Smilie.
Last edited by cmccabe; 01-17-2017 at 07:53 PM.. Reason: fixed format
# 2  
Old 01-18-2017
There is no creation date recorded for files (be they plain files, directories, pipes, sockets, devices, ..... etc) so it is unclear what results you will get. You are reading the modification time for the directory, i.e. the last time that a file was created/removed/renamed or had it's ownership/permissions changed and these are all going to be difficult to police.

What is the overall point of the process? Maybe we can consider a better way.



Robin
This User Gave Thanks to rbatte1 For This Post:
cmccabe
# 3  
Old 01-18-2017
The modification date is being used in the bash to determine the oldest file.

3 .tar.bz2 folders are downloaded and extracted by aria2cand the modification time of each folder is read by the bash.
That value starts with R_ always is stored as $filename but does not carry over the subsequent lines, so they don't execute.
Since $filename is written to the output log in the post, maybe it could be extracted? The problem is there could be multiple lines in that log, but only one $filename. That the variable needed for the subsequent lines. Thank you very much Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to store result in variable for other lines in script to use

I can not figure out how to capture the $filename variable store by the bash. #!/bin/bash # oldest folder stored as variable for analysis, version log created, and quality indicators matched to run dir=/home/cmccabe/Desktop/NGS/test find "$dir" -maxdepth 1 -mindepth 1 -type d -printf... (5 Replies)
Discussion started by: cmccabe
5 Replies

2. UNIX for Advanced & Expert Users

[BASH] Read value of variable, but not comment on the same line

Heyas So while i'm trying to increase security/usability of my TUI commands, i'm currently at tui-conf-get. There was also a bug, i wanted to fix, but sadly that bugfix is more insecure - but twice as fast as my current buggy method. I've added a getopts toggle, '-s' to toggle beSecure,... (8 Replies)
Discussion started by: sea
8 Replies

3. Shell Programming and Scripting

Nawk match regex of bash variable

Using a bash for loop to pass variables into a nawk loop to capture a string in an sftp log. Tried several different syntax methods to have the variable treated as a regex so the loop will capture the string. for i in `cat /tmp/dar3.out.2` do nawk -vst=$i '$5 ~ /$st/ && /closed/ && /user/... (3 Replies)
Discussion started by: smenago
3 Replies

4. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

5. Shell Programming and Scripting

Get line of textfile and store it in variable

Hi! I need to do the following: (1) I wan't to extract a line of a textfile (defined by a numer) and store it into a variable... (2) ...but I want to cut out a part of the line which is between two tokens and store just this to the variable Example: BlaBlaBla Bla2Bla2Bla2 *pPointerOne;... (4 Replies)
Discussion started by: Michi21609
4 Replies

6. Shell Programming and Scripting

Capturing a number at the end of line and store it as variable

Hello, Would someone guide me on how to write a shell script the would search for a phone no using at the end text file using sed or awk and store it in a varaible or print it. The text file is in this form text or numbers in first line text or numbers in second line . . . Firsname... (6 Replies)
Discussion started by: amuthiga
6 Replies

7. Shell Programming and Scripting

ksh: How to store each output line into a different variable?

Example output: /tmp/generatelines.sh line1 line2 line3 line4 I want each output line assigned to its own variable, ie: "line1" --> $a "line2" --> $b "line3" --> $c "line4" --> $d Is this possible without writing to a temporary file? Thanks (4 Replies)
Discussion started by: ksheller
4 Replies

8. Shell Programming and Scripting

bash script execution with a variable in a single line

Let a script needs a variable to execute. For example if i run ./test.sh then it needs a variable as there is a <STDIN> in the script. I want to execute it as in command line. Let test.sh requires a variable name $number I want to execute it by >test number <enter> how is it possible? (1 Reply)
Discussion started by: shoeb
1 Replies

9. Shell Programming and Scripting

store the first line of a file in a variable

i have to store the files in a folder and assign a variable to the the files. (0 Replies)
Discussion started by: dineshr85
0 Replies

10. UNIX for Dummies Questions & Answers

Read and store each line of a file to a variable

Hi all, I'm quite new to unix and hope that someone can help me on this. I'm using csh. Below is what i intend to do. 1. I stored some data in a file. 2. I intend to read the file line by line and store each line of data into a variable, so that i can used it later. Anyone have any... (4 Replies)
Discussion started by: seijihiko
4 Replies
Login or Register to Ask a Question