awk not working as expected in script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk not working as expected in script
# 1  
Old 10-24-2014
awk not working as expected in script

Dear all,
I had script which used to work, but recently it is not working as expected.
I have command line [1] in my shell script to choose the following format [2] from the output_elog [3] and perform some task afterwards on [2]

As you see, I want all numbers in foramt [2] following RED mark except for "Jobs with Wrapper Exit Code : 0".

Thank you,
emily

[1]
Code:
jobNo=$(awk '/Jobs with Wrapper/ && $NF != 0{s=1}   /List of jobs/ && s{if(p){p=p","$NF}else{p=$NF};s=""}END{print p}' output_elog)

[2]
Format is to pick up all numbers following the RED coded command_line in [3] but not BLUE coded, here I would expect to have the following output of [1]:
Code:
14,35,40,65,94,132,144,159,182,197,211,213,228,236,239,22,24-25,60,70,78,83,126,145,153,171,219,225,231,240,3,29,53,59,66,95,112-113,133,160,164,175,220,57,72,92,99,207,214,247,242,19,49,54,76,86,102,107,11\
0,119-120,181,210,221,234,5,63,97,204,246,71,117,84,215,50,208,10,114,229,87,148,158,166-168,244,9,30,51,61-62,69,127,150,157,165,174,177,238,243

[3] output_elog
Code:
crab:  ExitCodes Summary
 >>>>>>>>> 50 Jobs with Wrapper Exit Code : 50664
         List of jobs: 14,35,40,65,94,132,144,159,182,197,211,213,228,236,239,22,24-25,60,70,78,83,126,145,153,171,219,225,231,240,3,29,53,59,66,95,112-113,133,160,164,175,220,57,72,92,99,207,214,247
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 21 Jobs with Wrapper Exit Code : 60317
         List of jobs: 242,19,49,54,76,86,102,107,110,119-120,181,210,221,234,5,63,97,204,246,71
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 4 Jobs with Wrapper Exit Code : 50662
         List of jobs: 117,84,215,50
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 141 Jobs with Wrapper Exit Code : 0
         List of jobs: 1,6-8,13,15-18,20-21,23,26,33,39,42,44-48,55-56,58,64,68,79,81,93,96,98,100-101,106,109,115-116,118,121,123-125,128-131,134-143,146-147,149,151-152,154-156,161-162,170,173,176,178-179\
,183-194,196,198,200-203,205-206,209,216-217,230,233,235,237,241,31-32,36-37,43,82,91,111,122,2,4,11,27,38,52,67,73,75,80,85,103-105,108,169,172,199,222-223,227,232,12,74,88,163,218,226,245,248-249,41,195
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 1 Jobs with Wrapper Exit Code : 127
         List of jobs: 208
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 1 Jobs with Wrapper Exit Code : 10034
         List of jobs: 10
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:   250 Total Jobs
 >>>>>>>>> 9 Jobs Running
        List of jobs Running: 212,28,224,250,89,34,77,90,180
 >>>>>>>>> 210 Jobs Retrieved
        List of jobs Retrieved: 1,6-8,13,15-18,20-21,23,26,33,39,42,44-48,55-56,58,64,68,79,81,93,96,98,100-101,106,109,115-118,121,123-125,128-131,134-143,146-147,149,151-152,154-156,161-162,170,173,176,17\
8-179,183-194,196,198,200-203,205-206,209,216-217,230,233,235,237,241,14,31-32,35-37,40,43,65,82,91,94,111,122,132,144,159,182,197,211,213,228,236,239,242,2,4,11,19,22,24-25,27,49,52,54,60,67,70,73,75-76,78\
,80,83-86,102-105,107-108,110,119-120,126,145,153,169,171-172,181,199,210,215,219,221-223,225,227,231-232,234,240,3,5,12,29,50,53,59,63,66,74,88,95,97,112-113,133,160,163-164,175,204,218,220,226,245-246,248\
-249,10,41,71,195,208
 >>>>>>>>> 23 Jobs Cancelled
        List of jobs Cancelled: 114,229,87,148,158,166-168,244,9,30,51,61-62,69,127,150,157,165,174,177,238,243
 >>>>>>>>> 8 Jobs Done
        Jobs terminated: retrieve them with: crab -getoutput <List of jobs>
        List of jobs: 38,57,72,92,99,207,214,247

# 2  
Old 10-24-2014
This:
Code:
>>>>>>>>> 23 Jobs Cancelled

is not selected by your awk script. So try changing
Code:
/Jobs with Wrapper/

to
Code:
/Jobs (with Wrapper|Cancelled)/

and try again
# 3  
Old 10-24-2014
Hi, yes you are right..I did not include this cancelled category for now.
But the problem is when I try to print the jobID, its showing blank..just nothing..:-(

don't know, what is wrong with [1]..do you see any issues with [1] to perform this task (excluding cancelled category)?

thanks
emily
# 4  
Old 10-24-2014
Wrong answer
# 5  
Old 10-24-2014
Hello,
your answer is too short for me to understand what modification is needed in my script line [1].

thanks
emily
# 6  
Old 10-24-2014
I don't understand. This is your expected output:
Code:
14,35,40,65,94,132,144,159,182,197,211,213,228,236,239,22,24-25,60,70,78,83,126,145,153,171,219,225,231,240,3,29,53,59,66,95,112-113,133,160,164,175,220,57,72,92,99,207,214,247,242,19,49,54,76,86,102,107,110,119-120,181,210,221,234,5,63,97,204,246,71,117,84,215,50,208,10,114,229,87,148,158,166-168,244,9,30,51,61-62,69,127,150,157,165,174,177,238,243

The part in red is from the "Cancelled bit". When I execute:
Code:
awk '/Jobs (with Wrapper|Cancelled)/ && $NF != 0{s=1}   /List of jobs/ && s{if(p){p=p","$NF}else{p=$NF};s=""}END{print p}' output_elog

I get
Code:
14,35,40,65,94,132,144,159,182,197,211,213,228,236,239,22,24-25,60,70,78,83,126,145,153,171,219,225,231,240,3,29,53,59,66,95,112-113,133,160,164,175,220,57,72,92,99,207,214,247,242,19,49,54,76,86,102,107,110,119-120,181,210,221,234,5,63,97,204,246,71,117,84,215,50,208,10,114,229,87,148,158,166-168,244,9,30,51,61-62,69,127,150,157,165,174,177,238,243

# 7  
Old 10-24-2014
Hi,
yes you are right, thank you!

I just executed myself and found the expected answer...seems, issue is the way I get that output_elog, cos the script is not able to read it, as it seems!

I am checking that.

thank you,
emily
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Why this script is not working as 'expected' when doing ssh with while read ... really confused?

Hi, I have a script below that is running ssh <host> <command> on some servers. Below is more or less the script. I have to modify it somehow to get rid of the 'confidential' hostnames check_log.bash #!/bin/bash # myPID=$$ parse_log () { sub="parse_log" host=${1} ... (2 Replies)
Discussion started by: newbie_01
2 Replies

2. UNIX for Beginners Questions & Answers

Passing Arguments to shell script from file is not working as expected.

Hi All, I have below simple shell script in cloudera quick start vm cenos 6 which copy file from source to destination. # file_copy.sh source_dir = ${source_dir} target = ${target_dir} cp source_dir target and my parameter file is like below #parameter_file.txt source_dir =... (4 Replies)
Discussion started by: Narasimhasss
4 Replies

3. Shell Programming and Scripting

awk matching script not working as expected

This is my ubuntu version: $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial $ /bin/awk -V | head -n1 bash: /bin/awk: No such file or directory I have gotten a script that helps me to parse,... (14 Replies)
Discussion started by: delbroooks
14 Replies

4. Shell Programming and Scripting

awk gsub not working as expected

Hi Experts, Need your kind help with gsub awk. Below is my pattern:"exec=1_host_cnt=100_dup=4_NameTag=targetSrv_500.csv","'20171122112948"," 100"," 1"," 1"," 4","400","","", " aac sample exec ""hostname=XXXXX commandline='timeout 10 openssl speed -multi 2 ; exit 0'"" ","-1","-1","1","... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

5. Shell Programming and Scripting

awk command not working as expected

Following one line of awk code removes first 3 characters from each line but when I run the same code on another linux platform it doesn't work and only prints blank lines for each record. Can anyone please explain why this doesn't work? (31 Replies)
Discussion started by: later_troy
31 Replies

6. Shell Programming and Scripting

Script not working as expected

Hi, I have prepared a script and trying to execute it but not getting expected output. Could you please help and advise what is going wrong. "If else" part in below script is not working basically. I am running it on HP-UX. for i in slpd puma sfmdb do echo "******\t$i\t*******" echo... (10 Replies)
Discussion started by: sv0081493
10 Replies

7. Shell Programming and Scripting

bash variable (set via awk+sed) not working as expected

Hi! Been working on a script and I've been having a problem. I've finally narrowed it down to this variable I'm setting: servername=$(awk -v FS=\/ '{ print $7 } blah.txt | sed 's\/./-/g' | awk -v FS=\- '{print $1}')" This will essentially pare down a line like this: ... (7 Replies)
Discussion started by: creativedynamo
7 Replies

8. Shell Programming and Scripting

Var substitution in awk - not working as expected

countA=`awk '/X/''{print substr($0,38,1)}' fName | wc -l` countB=`wc -l fName | awk '{print int($1)}'` echo > temp ratio=`awk -va=$countA -vc=$countB '{printf "%.4f", a/c}' temp` After running script for above I am getting an error as : awk: 0602-533 Cannot find or open file -vc=25. The... (3 Replies)
Discussion started by: videsh77
3 Replies

9. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies

10. Shell Programming and Scripting

which not working as expected

Hello. Consider the following magic words: # ls `which adduser` ls: /usr/sbin/adduser: No such file or directory # Hmmm... Then: # ls /usr/sbin/adduser /usr/sbin/adduser # Now what? Unforunately this little sniippet is used in my debian woody server's mysql pre install script.... (2 Replies)
Discussion started by: osee
2 Replies
Login or Register to Ask a Question