Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

nawk is truncating output

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting nawk is truncating output
# 1  
Old 10-13-2012
nawk is truncating output
Legends,

I have 2 files f1 and f2. when i use nawk to compare the difference(subtraction) from 4th column of the file, it truncates the output.
can you please help to resolve this.

subtraction is (4th col of f1 - 4th col of f2). but it gives only below lines out of 116. I want to print all the lines of the file even if there is diff or no diff. Smilie

Code:
san:/tmp> wc -l f1 f2 | grep -v total
     116 f1
     116 f2

san:/tmp> head -3 f1 f2
==> f1 <==
TSCparser1 1irons1 EMEA_01 3
TSCparser12 1irons1 SPAIN_01 0
TSCparser13 1irons1 GERMANY_03 0

==> f2 <==
TSCparser1 1irons1 EMEA_01 3
TSCparser12 1irons1 SPAIN_01 0
TSCparser13 1irons1 GERMANY_03 0

san:/tmp> nawk 'FNR==NR{a[$1,$2,$3]=$4;next}{if(a[$1,$2,$3]){print $1,$2,$3,(a[$1,$2,$3]-$4)" times gapped in past 1 hr."}}' OFS="         " f1 f2
TSCparser1         1irons1         EMEA_01         0 times gapped in past 1 hr.
TSCparser94         1irons1         LSE_01         0 times gapped in past 1 hr.
TSCparser43         4irons1         STUTTGART_04         0 times gapped in past 1 hr.
TSCparser44         4irons1         STUTTGART_05         0 times gapped in past 1 hr.
TSCparser46         4irons1         STUTTGART_07         0 times gapped in past 1 hr.
TSCparser47         4irons1         STUTTGART_08         0 times gapped in past 1 hr.

# 2  
Old 10-13-2012
try this..

Code:
nawk 'FNR==NR{a[$1,$2,$3]=$4;next}{if(a[$1,$2,$3] != ""){print $1,$2,$3,(a[$1,$2,$3]-$4)" times gapped in past 1 hr."}}' OFS="\t" f1 f2

This User Gave Thanks to pamu For This Post:
sdosanjh
# 3  
Old 10-13-2012
The "error" is that in two of the three cases in your example, a[$1,$2,$3] exists, but is equal to zero. That's why awk won't print your line, even though the difference might be non-zero. Test it with $4 != 0 in f1. I'm not sure how to test the sheer existence of an entity in awk, but I think pamu has shown you a way to correct your statement.

-------------------------- edit ---------------------------------

 Reading man pages is educational. From the mawk man page:
Quote:
An expression, expr in array evaluates to 1 if array[expr] exists, else to 0.
so,
Code:
($1,$2,$3) in a {print $1,$2,$3,(a[$1,$2,$3]-$4)" tim..."}

will do the job.
Last edited by RudiC; 10-13-2012 at 01:26 PM.. Reason: test of sheer existence found
# 4  
Old 10-13-2012
Quote:
Originally Posted by sdosanjh
I want to print all the lines of the file even if there is diff or no diff.
Then, why are you checking something in if before printing the data? Drop that if:
Code:
nawk 'FNR==NR{a[$1,$2,$3]=$4;next}
{print $1,$2,$3,((($1,$2,$3) in a)?(a[$1,$2,$3]-$4):" ") " times gapped in past 1 hr."}' OFS="         " f1 f2

This will output all lines from f2. If matching line is found in f1, the numerical difference will be shown. Otherwise, a space will be shown in place of the difference.
Last edited by elixir_sinari; 10-13-2012 at 10:57 AM..
This User Gave Thanks to elixir_sinari For This Post:
sdosanjh
# 5  
Old 10-13-2012
You should also note that the value of SUBSEP varies in different implementations of awk (and I don't remember what value nawk uses). Some systems (for example OS X) default SUBSEP to an empty string. (SUBSEP is used to separate strings in multi-dimensional array subscripts). If there are any cases in your input where concatenating $1, $2, and $3 could yield a string that is not unique, you should explicitly set SUBSEP to something that doesn't appear in any of those three fields. Since $1 in your input ends with one or more digits and $2 starts with at least one digit, it looks like this could be possible issue with your input. For your input I would suggest setting SUBSEP to "," or "|" (e.g., add SUBSEP="," in your nawk command line after setting OFS).

RudiC said he didn't know how to test for the sheer existence of an entity in an array. The way to do that in this case would be to use:
Code:
if($1 SUBSEP $2 SUBSEP $3 in a) {...}

which would have the same meaning as:
Code:
if(a[$1,$2,$3] != "") {...}

in pamu's correction to the nawk script. In this case the test for an empty string is shorter than the test for existence (and for many is easier to read/understand), so I wouldn't make any change here.
This User Gave Thanks to Don Cragun For This Post:
sdosanjh
# 6  
Old 10-13-2012
Quote:
Originally Posted by Don Cragun
You should also note that the value of SUBSEP varies in different implementations of awk (and I don't remember what value nawk uses). Some systems (for example OS X) default SUBSEP to an empty string.
That's incorrect. From opensource.apple.com :: awk-18 :: tran.c (OS X 10.8.2):
Code:
char	**SUBSEP;	/* subscript separator for a[i,j,k]; default \034 */
...
SUBSEP = &setsymtab("SUBSEP", "\034", 0.0, STR|DONTFREE, symtab)->sval;

That code is also present in opensource.apple.com :: awk-1.2 :: tran.c (10.0), so it's not a recent change.

Any implementor who chooses an empty string for the value of SUBSEP should be shunned by the AWK community Smilie. Seriously, though, the chance for collisions would be too great.

OS X's awk is nawk (which is also used by the BSD systems). "\034" is also the value of SUBSEP in the mawk, GNU awk, and busybox awk implementations.

In light of this, fiddling with SUBSEP is usually unnecessary.

Regards,
Alister
Last edited by alister; 10-13-2012 at 12:02 PM..
These 3 Users Gave Thanks to alister For This Post:
Don Cragun Scrutinizer vgersh99
# 7  
Old 10-13-2012
Thanks Elixir and pamu, it is working now. the only thing i forgot to mention is f1 has higher numeric count than f2 always.
Code:
example: if in 1st run of script 4th col of f2 =123, and f1=125
then second run will be f2=125, f1=127, always greater than value in f2

Last edited by sdosanjh; 10-13-2012 at 02:03 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

How to avoid truncating in ps output ?

Hello, This is Solaris 10 (x86) bash-3.2# cat /etc/release Solaris 10 5/09 s10x_u7wos_08 X86 Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 30 March... (5 Replies)
Discussion started by: solaris_1977
5 Replies

2. UNIX for Dummies Questions & Answers

awk truncating first field output?

Hello, I'm writing an Awk script to take a command line argument (student's name) and output their relevant student#, name, and marks. For some reason, awk arbitrarily removes the first digit from the student number and doesn't show me the proper output. Here is my code: #! /usr/bin/awk -f... (6 Replies)
Discussion started by: trashmouth12
6 Replies

3. Shell Programming and Scripting

Nawk command to output in var

Hi I have this command, which counts number of lines in a specific file and then prints it on screen.nawk 'NF{c++}END{print "Number of GPS coordinates in file: "c}' $filename I would like to have the output put into a variable, but can't seem to find the correct argument for it. How do I... (3 Replies)
Discussion started by: bulleteyedk
3 Replies

4. Shell Programming and Scripting

Nawk Problem - nawk out of space in tostring on

Hi.. i am running nawk scripts on solaris system to get records of file1 not in file2 and find duplicate records in a while with the following scripts -compare nawk 'NR==FNR{a++;next;} !a {print"line"FNR $0}' file1 file2duplicate - nawk '{a++}END{for(i in a){if(a-1)print i,a}}' file1in the middle... (12 Replies)
Discussion started by: Abhiraj Singh
12 Replies

5. Shell Programming and Scripting

How to print and append output of nawk script in commandline and as well into a file?

Hi All, I am working on nawk script, has the small function which prints the output on the screen.Am trying to print/append the same output in a file. Basically nawk script should print the output on the console/screen and as well it should write/append the same result to a file. script :... (3 Replies)
Discussion started by: Optimus81
3 Replies

6. Shell Programming and Scripting

NAWK conversion of hexadecimal input to decimal output via printf, I am close I can feel it

I have searched and the answers I have found thus far have led me to this point, so I feel I am just about there. I am trying to convert a column of hexadecimal to decimal values so that I can filter out via grep just the data I want. I was able to pull my original 3 character hex value and... (10 Replies)
Discussion started by: PCGameGuy
10 Replies

7. Shell Programming and Scripting

help me how to use nawk for required output

Hi all i have 2 input files 1st file is N1 | N2|N3|N4|N5|N6|N7|N8|N9 4041491000|245160|1|0|0|1|0|0|0 4041401505|152178|1|1|1|1|0|0|0 4041450004|014052|1|1|1|1|0|0|0 4041450005|580916|1|1|1|1|0|0|0 4041491000|230990|1|0|1|1|0|0|0 4041460001|338317|1|1|1|1|0|0|0 2nd file N1 |... (8 Replies)
Discussion started by: dodasajan
8 Replies

8. Shell Programming and Scripting

Truncating a variable

I have a variable that is a full path name and I just want the file name with out the extension. I have figured out how to do this using some temp files but I would really like to avoid that if possible. I know I can do echo ${TMPNAME%.*} to drop the extension is there a similar way to drop... (3 Replies)
Discussion started by: whdr02
3 Replies

9. Shell Programming and Scripting

assigning nawk output to shell variable

Hello friends, I doing the follwing script , but found problem to store it to a shell variable. #! /bin/sh for temp in `find ./dat/vector/ -name '*.file'` do echo $temp nawk -v temp=$temp 'BEGIN{ split(temp, a,"\/"); print a}' done output: ./dat/vector/drf_all_002.file... (6 Replies)
Discussion started by: user_prady
6 Replies

10. Shell Programming and Scripting

Assigning nawk output to variables

I do a lot of command line scripting to capture data from files or other command output. I've checked in a number of Unix and scripting books but for the life of me I can't find out how to asign field data from nawk output into variables that I can manipulate later. For example, reading a two... (6 Replies)
Discussion started by: steveje0711
6 Replies
Login or Register to Ask a Question