Regexes for three column data to create a dictionary
I am working on a multilingual dictionary and I have data in three columns. The data structure can be
or
acts as a delimiter
The number of words separated by the delimiter can be up to 8 or 10. The structure is well defined in the sense that the number of words in the first column and the number of words in the second column are identical
An example will make this clear. For ease of comprehension I am using Latin script:
and so on.
I need to map the gloss in column3 to the string in column1 and the string in column 2
My query is how do I write a regex which will identify each of these types. Once I have the regex, I can write a script which will easily separate these out. I would appreciate a regex in Perl or Unix.
A script in either Perl or Awk would be the cherry on the cake. I work in a Windows environment
I hope to complete the mapper and put it up as a useful tool for multi-lingual transliteration across two languages. Many thanks.
perl does hashes really well. Is there a reason to use regexes? Once the hash is built in memory you could serialize it to disk, or more importantly you could use the hashtable instead of searching your file for each lookup. This will only benefit you if the file has a reasonably large number of lines. > few thousand
FWIW: how does your translator handle homographs? In English:lead a horse to water, the metal was as heavy as lead
Thanks for your comment. Insofar as homographs are concerned the gloss handles these as separate entries.
I am not very good at Perl and prefer to use Awk. If I am not asking too much how can this be handled in Perl.
However, most translators have lots of work to do, so however you do this you need to keep the hash in memory if performance is a criterion. So you may need to read through a bunch of "smallfile" with one read of bigfile -I don't know
You will need two field separators the "=" character and (for english tab and space).
The other possible field sep characters depend on your language.
files: bigfile (one with = separator) smallfile text to translate
I do not understand enough to give you a good answer. Some points:
1. the 'do your lookup here' part is simply an associative array lookup,
read $0 from smallfile, which should have the gloss, right?
2. you will probably need to create an awk function to lookup one[] and another for two[]
This User Gave Thanks to jim mcnamara For This Post:
Thanks a lot for your help with the script. You have grabbed what I have in mind. I will try and take on from here and suitably modify the script to accommodate other cases.
Looking at the samples in post#1, I'm not sure if the distinction between word1 and word2 (resulting in two arrays) is really needed. Wouldn't
do what you want?
Sorry for the late reply. I was out and could not access my mail. Many thanks for the answer.
I see your point that the distinction between word1 and word2 may not be needed.
I will test it out and get back to you.
Hi, I am newbie in shell script.
I need your help to solve my problem.
Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file.
File1:
SourceFile,DateTimeOriginal
/home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
I have a text file in UTF-8 format which has the following data structure
HEADWORD=gloss1,gloss2,gloss3 etc
I want to convert it so that all the glosses of the HeadWord appear on separate lines
HEADWORD=gloss1
HEADWORD=gloss2
HEADWORD=gloss3
An example will illustrate the requirement... (4 Replies)
Hello experts,
Please help me in achieving this in an easier way possible. I have 2 csv files with following data:
File1
08/23/2012 12:35:47,JOB_5330
08/23/2012 12:35:47,JOB_5330
08/23/2012 12:36:09,JOB_5340
08/23/2012 12:36:14,JOB_5340
08/23/2012 12:36:22,JOB_5350
08/23/2012... (5 Replies)
Hi,
I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column.
clientname,day-of-month,max-users
-----------------------------------
client1,20120610,5
client2,20120610,2
client3,20120610,7... (3 Replies)
Trying to understand what's happening here, but I cannot figure it out.
I'm reading Mastering Regular Expressions, by Friedl, and he uses this as an example of how to grab quoted text:
egrep -o '"*"' ~/File.txt
...should pull in any quoted phrases. Match a literal double-quote, match anything... (11 Replies)
1. The problem statement, all variables and given/known data:
Create a dictionary using cygwin. Display the following menu at the start of
execution
1-add a word in the dictionary # specify the meaning
2-search a word # if word exists, show the meaning of the word
2-delete a word... (2 Replies)
I have a data file that has a list of data macthing by user.
I am able to sort by user and there is multiple rows for each user.
Ideally I would like to email only the user of the files they own. Would it be best to create a seperate file by user and all rows showing the files they own? (9 Replies)
i am having a ms excel file which contains 2 columns, I first column i added words, and in second column meaning to the word in the first column. I want to create a dictionary for mobile like nokia or any java based application running mobile. How it can be created as i, dont know the java... (1 Reply)
Hello,
I am having a problem with the script I am using to create a column from two columns I have in my file. I am needing to take column 5 and subtract it from column 2 to create column 6. I have included the script I am using and the rawdata I am using.
Raw Data File:... (4 Replies)