Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Nawk Problem - nawk out of space in tostring on

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Nawk Problem - nawk out of space in tostring on
# 8  
Old 02-17-2014
Hi.

Suppose this is in file p1:
Code:
#!/usr/bin/perl
eval 'exec /usr/bin/perl -S $0 ${1+"$@"}'
    if $running_under_some_shell;
			# this emulates #! processing on NIH machines.
			# (remove #! line above if indigestible)

eval '$'.$1.'$2;' while $ARGV[0] =~ /^([A-Za-z_0-9]+=)(.*)/ && shift;
			# process any FOO=bar switches

$, = ' ';		# set output field separator
$\ = "\n";		# set output record separator

line: while (<>) {
    chomp;	# strip record separator
    if ($. == ($.-$FNRbase)) {
	$a{$_}++;
	next line;
    }
    if (!$a{$_}) {
	print 'line' . ($.-$FNRbase) . $_;
    }
}
continue {
    $FNRbase = $. if eof;
}

then try running it on a sample of your data to be sure it seems to do the right thing. Supply file names as you did for nawk (or possibly with the order reversed).

The Solaris box I have seem to have omiited processor a2p which automates the work of converting awk to perl. This was done on:
Code:
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian 5.0.8 (lenny, workstation) 
perl 5.10.0

Best wishes ... cheers, drl
# 9  
Old 02-17-2014
Have you tried using /usr/xpg4/bin/awk instead of nawk? I don't remember if there is much difference between those two versions of awk on Solaris systems in the way they handle memory management, but it might be worth a try if the perl script doesn't work for you.
# 10  
Old 02-17-2014
Correct your code, your double quote is mismatched also..
To compare files
Code:
nawk 'NR==FNR{a[$0];next;} !($0 in a){print "line:" FNR $0}' file1 file2

for duplicate try this
Code:
nawk '{A[$0]++}END{for(i in A)if(A[i]>1)print i,A[i]}' file

!a[$0] --> using a[$0] creates an extra empty array element for every $0 that does not exist in array a while reading the second file, so best thing is to do !($0 in a)

I really don't trust a[$0] to compare its my personal experience.
This User Gave Thanks to Akshay Hegde For This Post:
Abhiraj Singh
# 11  
Old 02-17-2014
Order remembered
Quote:
Originally Posted by jim mcnamara
Two suggestions -
1. gawk is available for Solaris 8->11 (URL OMMITED)
2. to find duplicates in a single file use an online method
Code:
  sort file | nawk 'NR==1 {old=$0; next}  {if (old==$0) {print $0}; old=$0}'

If you want to remember the original order, you may do something as this:
Code:
  nawk '{ print $0, length($0), NR }' file | sort | nawk 'NR==1 {old=substr($0, 1, $NF-1); next}  {if ( old==substr($0, 1, $NF-1) ) {print $0, $NF; old=substr($0, 1, $NF-1)}'

Just add the length and the original order to the records, sort, and then the last two fields is used to make the original record and display the original order.

Greetings!
# 12  
Old 02-17-2014
Hi.
Quote:
Originally Posted by Don Cragun
Have you tried using /usr/xpg4/bin/awk instead of nawk? I don't remember if there is much difference between those two versions of awk on Solaris systems in the way they handle memory management, but it might be worth a try if the perl script doesn't work for you.
That's a good point. The lengths are certainly different here:
Code:
$ ls -lig /usr/bin/awk /usr/bin/nawk /usr/xpg4/bin/awk /usr/bin/oawk
      7456 -r-xr-xr-x   2 bin        80184 Jan  8  2007 /usr/bin/awk
      7500 -r-xr-xr-x   1 bin       110100 Jan  8  2007 /usr/bin/nawk
      7456 -r-xr-xr-x   2 bin        80184 Jan  8  2007 /usr/bin/oawk
     35654 -r-xr-xr-x   1 bin        66816 Oct 10  2007 /usr/xpg4/bin/awk

but I have no idea about the internals. Note that oawk ("old awk") and awk are the same binary.

This system:
Code:
OS, ker|rel, machine: SunOS, 5.10, i86pc
Distribution        : Solaris 10 10/08 s10x_u6wos_07b X86

Best wishes ... cheers, drl
# 13  
Old 02-17-2014
Code:
nawk '{ print $0, length($0), NR }' file | sort | nawk 'NR==1 {old=substr($0, 1, $(NF-1) ); next}  {if ( old==substr($0, 1, $(NF-1) ) ) {print $0, $NF; old=substr($0, 1, $(NF-1))}'

Only to correct a mistake, the penultimate field is $(NF-1), not $NF-1; this is the last field minus one Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Nawk if logic problem

nawk '{ fmt="%3s %22s %48s %35s %21s\n"; if ($3==$6 && $1=="STOPLOSS") { tpy="Successful Match"; jnme=$1; sts="File will be loaded"; cntrl=$3; audit=$6; printf (fmt, tpy,jnme,sts,cntrl,audit) >> "'${AUDIT_DATA_FILE}/${AUDIT36}'" }else if ($3!=$6 && $1=="STOPLOSS") { tpy="Mis-Match ";... (4 Replies)
Discussion started by: wawa
4 Replies

2. Shell Programming and Scripting

problem in redirecting records using nawk

I am trying to redirect record to two files using nawk if-else. #Identify good and bad records and redirect records using if-then-else nawk -F"|" '{if(NF!=14){printf("%s\n",$0) >> "$fn"_bad_data}else{printf("%s\n",$0) >> $fn}}' "$fn".orig "$fn".orig is the source file name bad... (7 Replies)
Discussion started by: siteregsam
7 Replies

3. Shell Programming and Scripting

nawk and space in the parameter

Hi, Could you please tell me how nawk command works when there is a asterisk <*> or space with asterisk < *> or <* > in the parameter. I am just trying to read line by line and fetch fourth parameter separated by delimiter (|). But if there is a * or < *> or <* > in the fourth parameter it... (7 Replies)
Discussion started by: nram_krishna@ya
7 Replies

4. Shell Programming and Scripting

problem in nawk : case insensitive pattern matching

HI, My file contains data something like 034500,5,B5004946544EB185,DEFAULT,0 Now i want to do a pettern match for DEFAULT and remove that particular line from file and transfer the rest contents to temp file.But my req is i want to do case insensitive matching ie DEFAULT / default. I... (4 Replies)
Discussion started by: centurion_13
4 Replies

5. Shell Programming and Scripting

using nawk

help out with code. two files aaa bbb contains some records..output file xyz should be like this..see below i/p file:aaa 08350|60521|0000|505|0000|1555|000|NYCMT|Pd_1 |-11878 i/p file: bbb 60521|60510 o/p file :xyz 60510|08350|60521|0000|505|0000|1555|000|NYCMT|Pd_1 |-11878 (5 Replies)
Discussion started by: Diddy
5 Replies

6. Shell Programming and Scripting

Nesting - two nawk into one nawk

hi people; this is my two awk code: nawk '/cell+-/{r=(NF==8) ? $4FS$5FS$6 : NF==7 ? $4FS$5 : $4 ;c=split(r,rr);for (i=1;i<=c;i++){if(rr != "111111"){printf($3" %d ""\n",(i+3))}}printf("")}' /home/gc_sw/str.txt > /home/gc_sw/predwn.txt nawk -F'*' '{gsub(/ *$/,"")}$0=$1$($NF-2)'... (2 Replies)
Discussion started by: gc_sw
2 Replies

7. Shell Programming and Scripting

Multiple array problem in NAWK

I HAD these 2 files: file1 pictures.txt 5 ref2313 4 ref2345 3 ref5432 2 ref4244 1 dance.txt 6 ref2342 5 ref2352 4 ref0695 3 ref5738 2 ref4948 1 treehouse.txt 6 ref8573 5 ref3284 4 ref5838 3 ref4738 2 ref4573 1 file2 pictures.txt 1 3 dance.txt 2 4 treehouse.txt 3 5 what I... (1 Reply)
Discussion started by: linuxkid
1 Replies

8. Shell Programming and Scripting

Problem in splitiing file based on regex using awk/nawk

I have a file tmp.txt as shown below: Controller: 0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 1 Volume:c1t2d0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 2 Volume:c2t2d0 Disk: 0.2.0 Now I want to split... (4 Replies)
Discussion started by: durbam2002
4 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. Shell Programming and Scripting

nawk problem

How are ya, Heres the problem. I have a line of data that can either be in this format. "20" or "20kg" for the 20kg one i need to be able to read that their is kg on the end of this field and then ignore it and move on to the next line. Can anyone help. Cheers (3 Replies)
Discussion started by: ben_shark
3 Replies
Login or Register to Ask a Question