Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Why does this awk script not work correctly?
Top Forums Shell Programming and Scripting Why does this awk script not work correctly? Post 302996698 by gimley on Monday 1st of May 2017 03:00:57 AM
Old 05-01-2017
Sorry, I should have been more clear.
I work under Windows and hence DOS.
Basically as you can see the dictionary has a structure
Code:
English headword=Indic gloss,Indic gloss

as shown in the sample below:
Code:
10=दहा
10th=दहावा,दशम
11=अकरा,अकरा,एकादश
11th=अकरावा
12=बारा
12th=बारावा
13=तेरा,तेरा,त्रयोदश
13th=तेरावा
14=चौदा
14th=चौदावा
15=पंधरा
15th=पंधरावा,पंध्रावा
16=सोळा
16th=सोळावा
175=पावणेदोनशे,एकशे पंचाहत्तर
17=सतरा
17th=सतरावा
18=अठरा
18th=अठरावा
190=एकशेनव्वद
19=एकोणीस
19th=एकोणिसावा
1=एक
1st=प्रथम,पहिला
20=वीस
20th=विसावा
21=एकवीस
21st=एकविसावा
22=बावीस
22nd=बाविसावा
23=तेवीस
23rd=तेविसावा
24-hour interval=दिवस
24-karat gold=शुद्ध सोने,खरे सोने,अस्सल सोने,बावनकशी सोने

Since the database was made by hand at times, there are words repeated in the Indic glosses as shown in the sample below:
Code:
13=तेरा,तेरा,त्रयोदश
11=अकरा,अकरा,एकादश

What I needed was an awk script to identify such repeated entries and delete the duplicate entry.
Thus the sample above would be reduced as under
Code:
13=तेरा,त्रयोदश
11=अकरा,एकादश

I had written the following awk script to do the job:
Code:
# script to remove dupes from a row with structure word=word,word
BEGIN{FS="="}
{for(i=1;i<=NF;i++){a[$i]++;}for(i in a){b=b"="i}{sub("=","",b);$0=b;b="";delete a}}1

However when I ran the script on the sample, it produced a mangled output:
Code:
10=दहा
10th=दहावा,दशम
अकरा,अकरा,एकादश=11
11th=अकरावा
बारा=12
12th=बारावा
तेरा,तेरा,त्रयोदश=13
तेरावा=13th
चौदा=14
चौदावा=14th
पंधरा=15
15th=पंधरावा,पंध्रावा
सोळा=16
16th=सोळावा
पावणेदोनशे,एकशे पंचाहत्तर=175
17=सतरा
सतरावा=17th
18=अठरा
अठरावा=18th
एकशेनव्वद=190
19=एकोणीस
19th=एकोणिसावा
एक=1
प्रथम,पहिला=1st
वीस=20
विसावा=20th
एकवीस=21
21st=एकविसावा
बावीस=22
बाविसावा=22nd
तेवीस=23
23rd=तेविसावा
दिवस=24-hour interval
शुद्ध सोने,खरे सोने,अस्सल सोने,बावनकशी सोने=24-karat gold

I hope the above clarifies the situation. Identifying dupes visually is both time-consuming and prone to error.

---------- Post updated at 02:00 AM ---------- Previous update was at 01:57 AM ----------

By the time I had posted the clarifications, you had already replied. Many thanks, it worked and swept through a dictionary of 70,000 words and removed all the dupes.
I will now study the script to see where I went wrong
 

10 More Discussions You Might Find Interesting

1. SuSE

vsft doesn't work correctly

I install vsftpd server on 2 SUSE 10.2 servers. The first works perfectly, but the second doesn't work how I expect. The second works only over local network and doesn't over internet. The vsftpd.conf and ../xinetd.d/vsftpd are the same in 2 servers. The only different was when I threw to log in... (1 Reply)
Discussion started by: zhivko.neychev
1 Replies

2. Shell Programming and Scripting

Awk: first line not processed correctly

Hey, I've made a little awk-script which reorders lines. Works okay, only problem is that is doesn't process the first line correctly. If I switch lines in the Input file it doesn't proces this first line either. Somebody please help! Here's is the code and the input file! thanx ... (1 Reply)
Discussion started by: BartleDuc
1 Replies

3. Shell Programming and Scripting

Grep/awk not getting the message correctly

I have a script which will take two file as the inputs and take the Value in file1 and search in file2 and give the output in Outputfile. #!/bin/sh #. ${HOME}/crossworlds/bin/CWSharedEnv.sh FILE1=$1 FILE2=$2 for Var in $(cat $FILE1);do echo $Var grep -i "$Var" $FILE2 done > Outputfile I... (2 Replies)
Discussion started by: SwapnaNaidu
2 Replies

4. Shell Programming and Scripting

awk script to remove spaces - examples don't show up correctly

I have the following data from a manual database dump. I need to format the columns so that I can import them into an excel spread sheet. So far I have been able to get past the hurdles with vi and grep. Now I have one last issue that I can't get past. Here is an example of the data. Here is... (18 Replies)
Discussion started by: Chris_Rivera
18 Replies

5. Programming

Cannot get dbx to work correctly with a running process

Hi everyone, I've been struggling with this for a few weeks now. I'm trying to debug a running process with dbx on an AIX box. The command I'm using is 'dbx -a <pid> core' There is a function I can perform in my application that crashes this process, but it does not show up as crashed in... (0 Replies)
Discussion started by: ctote
0 Replies

6. Shell Programming and Scripting

awk not working correctly

Hi I am attempting to right a script which will read a table and extract specfic information. LASTFAILEDJOB=/usr/openv/netbackup/scripts/GB-LDN/Junaid/temp_files/lastfailedjob cat /usr/openv/netbackup/scripts/GB-LDN/Junaid/temp_files/lastfailedjob 237308646 If i run the following... (5 Replies)
Discussion started by: Junes
5 Replies

7. UNIX for Advanced & Expert Users

Libvirt does not work correctly anymore on my gentoo

Hi, Since a year my libvirtd does not work anymore on my Gentoodesktop. In the meantime a used virtualbox. But I would like to have back libvirt. The problem was after libvirt should not only work with root privileges. I deinstalled all things with libvirt an kvm. I removed all things from /var... (4 Replies)
Discussion started by: darktux
4 Replies

8. SCO

Set NIC correctly , but the network does not work

I'm trying to virtualize an instance of SCO Open Server 5.0.2c in VirtualBox (called VM- A) , I can not configure the network (NIC). The NIC I'm using is PCnet -FAST III (Am79C973 ) (this NIC works with VirtualBox + SCO 5.0.5M) When I add from ' Add new LAN adapter' I detects the NIC... (2 Replies)
Discussion started by: flako
2 Replies

9. Gentoo

LDAP-Auth does not work correctly with systemd

Hi, since the upgrade to Gnome 3.6 (now i have 3.8) the authentication over LDAP stops working. The whole machine does not start anymore. The machine boot, but no gdm and no X. I can login, with root, but then the tty hangs. When i look at ttyF12 i see a lot of systemd service the runs random,... (1 Reply)
Discussion started by: darktux
1 Replies

10. Shell Programming and Scripting

Work with setsid to open a session and close it correctly

I would like to create the following script: run a python script with setsid python may or may not fail with exception check if all of the group processes were terminated correctly if not, kill the remaining processes How can I do that? Thanks a lot (3 Replies)
Discussion started by: ASF Studio
3 Replies
All times are GMT -4. The time now is 03:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy