CREATING A SYLLABLE CONCORDANCE WITH POSITIONAL VARIANTS Post: 302543707

Sponsored Content

Top Forums Shell Programming and Scripting CREATING A SYLLABLE CONCORDANCE WITH POSITIONAL VARIANTS Post 302543707 by gimley on Monday 1st of August 2011 10:46:36 PM

08-01-2011

Registered User

Hello,
With a little help from colleagues, I finally managed to get the concordance going. Here is the code in case someone else would like to use it:

Code:

#! /usr/bin/perl

use strict;  # These two lines save you endless trouble
use warnings; # without them typos and such errors get missed

open (my $corpus_file, '<', 'Corpus'); # Created a test corpus with just the contained lines
# $/="\r\n"; # Again with the DOS files
chomp(my @corpus = (<$corpus_file>)); # Load the corpus file into an array for faster access
open (my $syllables_file, '<', 'Syllables');
while(<$syllables_file>){
    chomp(my $syllable = $_);
    my $count = 0;
    my $init = my $med = my $fin = my $stdalone = "NONE";
    for my $word (@corpus) {
        if ( $word =~ /^$syllable.+/) {
            if ($init eq "NONE") {
                $init = $word;
                $count++;
            }
        }
        elsif ($word =~ /.+$syllable.+/) {
            if ($med eq "NONE") {
                $med = $word;
                $count++;
            }
        }
        elsif ($word =~ /.+$syllable$/) {
            if ($fin eq "NONE") {
                $fin = $word;
                $count++;
            }
        }
        elsif ($word =~ /^$syllable$/) {
            if ($stdalone eq "NONE") {
                $stdalone = $word;
                $count++;
            }
        }
        last if $count == 4;
    }
    print "$syllable\nInitial $init\nMedial $med\nFinal $fin\nStandalone $stdalone\n";
    #print "$init\t$med\t$fin\t$stdalone\n";
}

Many thanks for the information re. Regex.

Last edited by Scott; 08-02-2011 at 01:14 AM.. Reason: Code tags, please...

gimley

View Public Profile for gimley

Find all posts by gimley

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Creating a syllable concordance

Hello, I have two files. The first file contains specific syllables of a language (Hindi) and the second file contains a large database from which these syllables have been culled. The syllable file which has syllables in Hindi has one syllable per line and the corpus file has a data...

2. Shell Programming and Scripting

[All variants] remove first pair of parentheses

How to remove first pair of parentheses and content in them from the beginning of the line? Here's the list: (ok)-test (ok)-test-(ing) (some)-test-(ing)-test test-(ing) Desired result: test test-(ing) test-(ing)-test test-(ing) Here's what I already tried with GNU sed: sed -e...

3. Shell Programming and Scripting

Writing a clustering concordance for a Perso-Arabic script

I am working on a database of a language using Arabic Script. One of the major issues is that the shape of the characters changes according to their initial, medial or final positioning. Another major issue is that of the clustering of vowels within the word: the clustering changes totally the...

4. Shell Programming and Scripting

[All variants] Change settings

Hi, I have a big settings confg (file attached). There are a few separate tasks that I have to accomplish. All scripting/programming languages are appreciated. 1. I need to parse all values and output to stdout. Sample output (truncated): VALUEA 2017-01-01 Lores ipsum Lorem ipsum dolor sit...

5. UNIX for Beginners Questions & Answers

Merge 4 bim files by keeping only the overlapping variants (unique rs values )

Dear community, I am facing a problem and I kindly ask your help: I have 4 different data sets consisted from 3 different types of array. On each file, column 1 is chromosome position, column 2 is SNP id etc... Lets say I have the following (bim) datasets: x2014: 1 rs3094315...

LEARN ABOUT DEBIAN

mmseg

MMSEG(1)						User Contributed Perl Documentation						  MMSEG(1)

NAME

       mmseg - maximum matching segment Chinese text.

SYNOPSIS

       mmseg -d dict_file [option]... [corpus_file]...

DESCRIPTION

       mmseg is a tool for segmenting Chinese text into words using maximum matching algorithm. mmseg segments corpus_file, or standard input if
       no filename is specified, and write the segmented result to standard output.

OPTIONS

       -d dict_file
	   Use dict_file as lexicon. A default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8.

       -f,--format (text|bin)
	   Output Format, can be 'text' or 'bin'. default 'bin'.  Normally, in text mode, word text are output, while in binary mode, binary short
	   integer of the word-ids are written to stdout.

       -s, --stok STOK_ID
	   Sentence token id. Default 10.  It will be written to output in binary mode after every sentence.

       -i, --show-id
	   Show Id info. Under text output format mode, attach id after known words.  If under binary mode, print id(s) in text.

       -a, --ambiguious-id AMBI-ID
	   Ambiguious means ABC => A BC or AB C. If specified (AMBI-ID != 0), The sequence ABC will not be segmented, in binary mode, the AMBI-ID
	   is written out; in text mode, "<ambi>ABC</ambi>" will be output. Default is 0.

NOTES

       Under binary mode, consecutive id of 0 are merged into one 0.  Under text mode, no space are inserted between unknown-words.

AUTHOR

       Originally written by Phill.Zhang <phill.zhang@sun.com>.  Currently maintained by Kov.Chai <tchaikov@gmail.com>.

SEE ALSO

       slmseg(1), ids2ngram (1).

perl v5.14.2							    2012-06-09								  MMSEG(1)