Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk not working as expected in script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk not working as expected in script
# 8  
Old 10-24-2014
Quote:
Originally Posted by emily
Hello,
your answer is too short for me to understand what modification is needed in my script line [1].

thanks
emily
I modified my post, because I understood you incorrectly. Sorry about that.
# 9  
Old 10-24-2014
Hi again,
I am still puzzeled as its not working [4], though that script line t is all OK. The output of this [4] is following [5] as I checked. And I checked that the output of the GEN-SIM-DIGI-L1_v2_010813_status.log is [6],. which is all OK..:-(

Maybe I am making some stupid mistake,if anybody spot it ...please let me know.

thank in advance,
emily


[4]
Code:
crab -getoutput -c $SAMPLE
    crab -status -c $SAMPLE >& $SAMPLE"_status.log" &

    echo $CODEPATH/$SAMPLE"_status.log" "====="
    jobNo=$(awk '/Jobs with Wrapper/ && $NF != 0{s=1}   /List of jobs/ && s{if(p){p=p","$NF}else{p=$NF};s=""}END{print p}' "$SAMPLE"_status.log)
    sleep 50
    echo "The present working directory is `pwd`"
    echo ">>>>>>>>>>>>>>>>>>>>> JObs SUBMITTED ARE FOLLOWING: " $jobNo
    echo "DONE"

[5]
Code:
 /nfs/dust/cms/user/emily/scratch/hadron_outer/emulation_HO/CMSSW_7_1_0/src//L1Trigger/L1IntegratedMuonTrigger/test/GEN-SIM-DIGI-L1_v2_010813_status.log =====
The present working directory is /nfs/dust/cms/user/emily/scratch/hadron_outer/emulation_HO/CMSSW_7_1_0/src/L1Trigger/L1IntegratedMuonTrigger/test
>>>>>>>>>>>>>>>>>>>>> JObs SUBMITTED ARE FOLLOWING: 

DONE

[6]
Code:
 crab:  ExitCodes Summary
 >>>>>>>>> 2 Jobs with Wrapper Exit Code : 60317
         List of jobs: 207,250
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 85 Jobs with Wrapper Exit Code : 0
         List of jobs: 5,36-37,57,88,104,126,131,137-138,141,143,146,148,160,179-180,185,196,214-216,223-224,229,233,237-238,248-249,54,90,97,115-117,129,167,173,176,189,239,245,158,1,19,38,41,43,52-53,67,8\
0,82-84,105,108,110,112,120,124,136,139,142,147,150,170,186,227,230,234,6,9-11,16,45,85,101,106,194,201-202,205
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:  ExitCodes Summary
 >>>>>>>>> 2 Jobs with Wrapper Exit Code : 50664
         List of jobs: 236,190
        See https://twiki.cern.ch/twiki/bin/view/CMS/JobExitCodes for Exit Code meaning

crab:   250 Total Jobs
 >>>>>>>>> 131 Jobs Running
        List of jobs Running: 134,187,206,2-3,8,12-13,15,17-18,20,22,25,27-30,32-33,35,39-40,42,44,46-51,55,58-66,68-76,78,86-87,89,91-96,98-100,102-103,107,109,111,113-114,118-119,121-123,125,127-128,130,1\
32-133,144-145,151,153,155-157,159,161-164,166,168-169,171-172,174-175,177-178,181-183,191-193,195,203-204,208-209,211-213,217,219-222,225-226,228,231-232,235,241,244,246-247
 >>>>>>>>> 1 Jobs Aborted
        You can resubmit them specifying JOB numbers: crab -resubmit <List of jobs>
        List of jobs: 218
 >>>>>>>>> 90 Jobs Retrieved
        List of jobs Retrieved: 5,36-37,57,88,104,126,131,137-138,141,143,146,148,160,179-180,185,196,198,214-216,223-224,229,233,237-238,248-249,54,90,97,115-117,129,167,173,176,189,236,239,245,158,207,1,1\
9,38,41,43,52-53,67,80,82-84,105,108,110,112,120,124,136,139,142,147,150,170,186,227,230,234,250,6,9-11,16,45,85,101,106,194,201-202,190,205
 >>>>>>>>> 28 Jobs Cancelled
        List of jobs Cancelled: 7,14,21,23,34,135,140,149,152,154,165,197,210,240,242,4,24,26,31,56,77,79,81,184,188,243,199-200

# 10  
Old 10-24-2014
You did not adjust the code? The same line seems to be used.

Another thing I noticed is that on this line:
Code:
crab -status -c $SAMPLE >& $SAMPLE"_status.log" &

The command is executed in the background and is writing to a file.
On the second line after that, line the file is being read however., which should only happen after the background job has finished.

So either insert a wait statement between those two lines, or remove the closing & of the first line and thus run it in the foreground.
These 2 Users Gave Thanks to Scrutinizer For This Post:
emily Kibou
# 11  
Old 10-24-2014
Hi,
I put sleep 50, should not that be OK?
Even I increase it to sleep 200, it does not help..

emily
# 12  
Old 10-24-2014
Quote:
Originally Posted by emily
Hi,
I put sleep 50, should not that be OK?
Even I increase it to sleep 200, it does not help..

emily
emily did you ever watch Jerry Maguire?: "Help me help you!!"

Scrutinizer asked: "You did not adjust the code? The same line seems to be used."
You have not addressed that.

Have you tried wait as Scrutinizer mentioned?
Have you tried to remove the process from background?

It is OK to ignore any advise you receive after you asked for, nevertheless, it would be proper to explain why if you want to continue asking.
This User Gave Thanks to Aia For This Post:
Scrutinizer
# 13  
Old 10-25-2014
Scrutinizer- thanking you so much for your help. And sorry for posting hurriedly without trying your advice.

Greetings,
emily
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Why this script is not working as 'expected' when doing ssh with while read ... really confused?

Hi, I have a script below that is running ssh <host> <command> on some servers. Below is more or less the script. I have to modify it somehow to get rid of the 'confidential' hostnames check_log.bash #!/bin/bash # myPID=$$ parse_log () { sub="parse_log" host=${1} ... (2 Replies)
Discussion started by: newbie_01
2 Replies

2. UNIX for Beginners Questions & Answers

Passing Arguments to shell script from file is not working as expected.

Hi All, I have below simple shell script in cloudera quick start vm cenos 6 which copy file from source to destination. # file_copy.sh source_dir = ${source_dir} target = ${target_dir} cp source_dir target and my parameter file is like below #parameter_file.txt source_dir =... (4 Replies)
Discussion started by: Narasimhasss
4 Replies

3. Shell Programming and Scripting

awk matching script not working as expected

This is my ubuntu version: $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial $ /bin/awk -V | head -n1 bash: /bin/awk: No such file or directory I have gotten a script that helps me to parse,... (14 Replies)
Discussion started by: delbroooks
14 Replies

4. Shell Programming and Scripting

awk gsub not working as expected

Hi Experts, Need your kind help with gsub awk. Below is my pattern:"exec=1_host_cnt=100_dup=4_NameTag=targetSrv_500.csv","'20171122112948"," 100"," 1"," 1"," 4","400","","", " aac sample exec ""hostname=XXXXX commandline='timeout 10 openssl speed -multi 2 ; exit 0'"" ","-1","-1","1","... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

5. Shell Programming and Scripting

awk command not working as expected

Following one line of awk code removes first 3 characters from each line but when I run the same code on another linux platform it doesn't work and only prints blank lines for each record. Can anyone please explain why this doesn't work? (31 Replies)
Discussion started by: later_troy
31 Replies

6. Shell Programming and Scripting

Script not working as expected

Hi, I have prepared a script and trying to execute it but not getting expected output. Could you please help and advise what is going wrong. "If else" part in below script is not working basically. I am running it on HP-UX. for i in slpd puma sfmdb do echo "******\t$i\t*******" echo... (10 Replies)
Discussion started by: sv0081493
10 Replies

7. Shell Programming and Scripting

bash variable (set via awk+sed) not working as expected

Hi! Been working on a script and I've been having a problem. I've finally narrowed it down to this variable I'm setting: servername=$(awk -v FS=\/ '{ print $7 } blah.txt | sed 's\/./-/g' | awk -v FS=\- '{print $1}')" This will essentially pare down a line like this: ... (7 Replies)
Discussion started by: creativedynamo
7 Replies

8. Shell Programming and Scripting

Var substitution in awk - not working as expected

countA=`awk '/X/''{print substr($0,38,1)}' fName | wc -l` countB=`wc -l fName | awk '{print int($1)}'` echo > temp ratio=`awk -va=$countA -vc=$countB '{printf "%.4f", a/c}' temp` After running script for above I am getting an error as : awk: 0602-533 Cannot find or open file -vc=25. The... (3 Replies)
Discussion started by: videsh77
3 Replies

9. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies

10. Shell Programming and Scripting

which not working as expected

Hello. Consider the following magic words: # ls `which adduser` ls: /usr/sbin/adduser: No such file or directory # Hmmm... Then: # ls /usr/sbin/adduser /usr/sbin/adduser # Now what? Unforunately this little sniippet is used in my debian woody server's mysql pre install script.... (2 Replies)
Discussion started by: osee
2 Replies
Login or Register to Ask a Question