Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Script to create unique look-up for headers for a Dictionary

Shell Programming and Scripting


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 04-19-2013   -   Original Discussion by gimley
gimley gimley is offline
Registered User
 
Join Date: Feb 2011
Last Activity: 5 November 2017, 10:40 PM EST
Posts: 280
Thanks: 131
Thanked 3 Times in 3 Posts
Script to create unique look-up for headers for a Dictionary

I have a text file in UTF-8 format which has the following data structure

Code:
HEADWORD=gloss1,gloss2,gloss3 etc

I want to convert it so that all the glosses of the HeadWord appear on separate lines

Code:
HEADWORD=gloss1
HEADWORD=gloss2
HEADWORD=gloss3

An example will illustrate the requirement
INPUT

Code:
હોશમાં આવવું=regain consciousness.
હોશિયાર=clever, intelligent; skilful; alert, vigilant; cautious; understanding, sensible.
હોશિયારી કરવી=boast,(try to) be clever.
હોશિયારી દાખવવી=boast,(try to) be clever.
હોશિયારી બતાવવી=boast,(try to) be clever.
હોશિયારી મારવી=boast,(try to) be clever.
હોશિયારી રાખવી=be cautious,be vigilant,be alert.
હોશિયારી=cleverness, vigilance
હોહા=noise, uproar, tumult, public talk or discussion, excitement, agitation, alarm, consternation.
હોહાકાર=uproar, tumult, excitement, alarm.
હોહો=noise, uproar, tumult, public talk or discussion, excitement, agitation, alarm, consternation.

The Output would be

Code:
હોશિયાર=clever
હોશિયાર=intelligent
હોશિયાર=skilful
હોશિયાર=alert
હોશિયાર=vigilant
હોશિયાર=cautious
હોશિયાર=understanding
હોશિયાર=sensible.
હોશિયારી કરવી=boast
હોશિયારી કરવી=(try to) be clever.
હોશિયારી દાખવવી=boast
હોશિયારી દાખવવી=(try to) be clever.
હોશિયારી બતાવવી=boast
હોશિયારી બતાવવી=(try to) be clever.
હોશિયારી મારવી=boast
હોશિયારી મારવી=(try to) be clever.
હોશિયારી રાખવી=be cautious
હોશિયારી રાખવી=vigilant or alert.
હોશિયારી=cleverness
હોશિયારી=vigilance
હોશિયારી=etc.
હોહા=noise
હોહા=uproar
હોહા=tumult
હોહા=public talk or discussion
હોહા=excitement
હોહા=agitation
હોહા=alarm
હોહા=consternation.
હોહાકાર=uproar
હોહાકાર=tumult
હોહાકાર=excitement
હોહાકાર=alarm
હોહો=noise
હોહો=uproar
હોહો=tumult
હોહો=public talk or discussion
હોહો=excitement
હોહો=agitation
હોહો=alarm
હોહો=consternation

At present I use macros which identify the delimiter, copy the text between two delimiters, paste it on next line, preface it with the headword and continue the operation till end of line and repeat the same for the next line. Since the file is huge a PERL or AWK script would help.
I work under Windows and UNIX type solutions do not work for me unfortunately.
Many thanks in advance.
Sponsored Links
    #2  
Old Unix and Linux 04-19-2013   -   Original Discussion by gimley
clx clx is offline Forum Advisor  
Registered User
 
Join Date: Jun 2007
Last Activity: 21 November 2017, 8:11 AM EST
Location: Mumbai, India
Posts: 1,690
Thanks: 138
Thanked 205 Times in 199 Posts
May be this?

Code:
echo HEADWORD=gloss1,gloss2,gloss3 | awk -F "[,=]" '{for (i=2;i<=NF;i++) {$i=$1"="$i;print $i}}'

Sponsored Links
    #3  
Old Unix and Linux 04-19-2013   -   Original Discussion by gimley
gimley gimley is offline
Registered User
 
Join Date: Feb 2011
Last Activity: 5 November 2017, 10:40 PM EST
Posts: 280
Thanks: 131
Thanked 3 Times in 3 Posts
Hello,
Many thanks for your help.
I tried the script you provided but got the following output on the sample file:

Code:
હોશમાં આવવું=regain consciousness.
હોશમાં=આવવું=regain
હોશમાં=consciousness.
હોશિયાર=clever, intelligent; skilful; alert, vigilant; cautious; understanding, sensible.
હોશિયાર=clever,=intelligent;
હોશિયાર=clever,=skilful;
હોશિયાર=clever,=alert,
હોશિયાર=clever,=vigilant;
હોશિયાર=clever,=cautious;
હોશિયાર=clever,=understanding,
હોશિયાર=clever,=sensible.
હોશિયારી કરવી=boast,(try to) be clever.
હોશિયારી=કરવી=boast,(try
હોશિયારી=to)
હોશિયારી=be
હોશિયારી=clever.
હોશિયારી દાખવવી=boast,(try to) be clever.
હોશિયારી=દાખવવી=boast,(try
હોશિયારી=to)
હોશિયારી=be
હોશિયારી=clever.
હોશિયારી બતાવવી=boast,(try to) be clever.
હોશિયારી=બતાવવી=boast,(try
હોશિયારી=to)
હોશિયારી=be
હોશિયારી=clever.
હોશિયારી મારવી=boast,(try to) be clever.
હોશિયારી=મારવી=boast,(try
હોશિયારી=to)
હોશિયારી=be
હોશિયારી=clever.
હોશિયારી રાખવી=be cautious,be vigilant,be alert.
હોશિયારી=રાખવી=be
હોશિયારી=cautious,be
હોશિયારી=vigilant,be
હોશિયારી=alert.
હોશિયારી=cleverness, vigilance
હોશિયારી=cleverness,=vigilance
હોહા=noise, uproar, tumult, public talk or discussion, excitement, agitation, alarm, consternation.
હોહા=noise,=uproar,
હોહા=noise,=tumult,
હોહા=noise,=public
હોહા=noise,=talk
હોહા=noise,=or
હોહા=noise,=discussion,
હોહા=noise,=excitement,
હોહા=noise,=agitation,
હોહા=noise,=alarm,
હોહા=noise,=consternation.
હોહાકાર=uproar, tumult, excitement, alarm.
હોહાકાર=uproar,=tumult,
હોહાકાર=uproar,=excitement,
હોહાકાર=uproar,=alarm.
હોહો=noise, uproar, tumult, public talk or discussion, excitement, agitation, alarm, consternation.
હોહો=noise,=uproar,
હોહો=noise,=tumult,
હોહો=noise,=public
હોહો=noise,=talk
હોહો=noise,=or
હોહો=noise,=discussion,
હોહો=noise,=excitement,
હોહો=noise,=agitation,
હોહો=noise,=alarm,
હોહો=noise,=consternation.

Did I cut and paste wrongly the code.
Thanks again but it did not work.
    #4  
Old Unix and Linux 04-19-2013   -   Original Discussion by gimley
itkamaraj's Unix or Linux Image
itkamaraj itkamaraj is offline Forum Advisor  
Registered User
 
Join Date: Apr 2010
Last Activity: 5 November 2017, 9:59 PM EST
Posts: 3,148
Thanks: 45
Thanked 698 Times in 674 Posts
wait....

---------- Post updated at 08:30 PM ---------- Previous update was at 08:18 PM ----------


Code:
$ awk -F= '{a=$1;sub(a"=","");$0=$0;FS="[;,=]";for(i=2;i<=NF;i++)print a" ="$i}' a.txt
હોશિયાર = intelligent
હોશિયાર = skilful
હોશિયાર = alert
હોશિયાર = vigilant
હોશિયાર = cautious
હોશિયાર = understanding
હોશિયાર = sensible.
હોશિયારી કરવી =(try to) be clever.
હોશિયારી દાખવવી =(try to) be clever.
હોશિયારી બતાવવી =(try to) be clever.
હોશિયારી મારવી =(try to) be clever.
હોશિયારી રાખવી =be vigilant
હોશિયારી રાખવી =be alert.
હોશિયારી = vigilance
હોહા = uproar
હોહા = tumult
હોહા = public talk or discussion
હોહા = excitement
હોહા = agitation
હોહા = alarm
હોહા = consternation.
હોહાકાર = tumult
હોહાકાર = excitement
હોહાકાર = alarm.
હોહો = uproar
હોહો = tumult
હોહો = public talk or discussion
હોહો = excitement
હોહો = agitation
હોહો = alarm
હોહો = consternation.

The Following User Says Thank You to itkamaraj For This Useful Post:
gimley (04-23-2013)
Sponsored Links
    #5  
Old Unix and Linux 04-19-2013   -   Original Discussion by gimley
gimley gimley is offline
Registered User
 
Join Date: Feb 2011
Last Activity: 5 November 2017, 10:40 PM EST
Posts: 280
Thanks: 131
Thanked 3 Times in 3 Posts
Many thanks for the script. It works beautifully and handled 50,000 dictionary entries in no time
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Create shell script to extract unique information from one file to a new file. Mr_47 Shell Programming and Scripting 16 09-11-2011 02:33 PM
How to create a dictionary using cygwin kpopfreakghecky Homework & Coursework Questions 2 08-30-2011 07:03 AM
How to create java based dictionary for mobile using data in microsoft excel? Anna Hussie Programming 1 04-11-2011 05:27 PM
Merging of files with different headers to make combined headers file marut_ashu Shell Programming and Scripting 1 08-07-2009 05:10 AM
Remove text between headers while leaving headers intact Trones Shell Programming and Scripting 3 12-27-2008 08:06 PM



All times are GMT -4. The time now is 02:26 PM.