Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Deleting duplicate glosses in a dictionary entry
Top Forums Shell Programming and Scripting Deleting duplicate glosses in a dictionary entry Post 302844697 by balajesuri on Monday 19th of August 2013 09:28:40 PM
Old 08-19-2013
Here's a perl program, though, I couldn't test it with the actual data (urdu and hindi characters). It works for ASCII characters input (a=b,c,b.......)
Code:
#! /usr/bin/perl

use warnings;
use strict;

my ($line, @lr, %hindi_words);
open I, "< file.txt";
while ($line = <I>) {
    chomp ($line);
    undef %hindi_words;
    @lr = split ('=', $line);
    for (split(',', $lr[1])) {
        $hindi_words{$_} = 1;
    }
    print "$lr[0]=", join(',', keys(%hindi_words)), "\n";
}
close I;

By the way, for this program logically similar words like आबादिओं,आबादियों or आज कल,आजकल or ऑबजेक्शन,ऑब्जेक्शन are different.
This User Gave Thanks to balajesuri For This Post:
gimley
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting double entry in a file

Hi, I am having almost the same problem as junior member 'oupsforum' (refer to subjuct "deleting double entry in a log file"), only that I am using Sun Sorlaris Unix which the uniq command does not has the flag -w. So I am not able to ignore certain portion of the line when the uniq doing the... (3 Replies)
Discussion started by: Wing m. Cheng
3 Replies

2. HP-UX

Hazardous Duplicate Cron Entry?

Hi All, How to prevent starting of processes that have duplicate entries in cron file, i have written a shell script to validate with "ps |grep" command before starting the process, but still when same process started at same time, it may not be able to detect the existing process. Sample... (3 Replies)
Discussion started by: nag_sundaram
3 Replies

3. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge... (5 Replies)
Discussion started by: kharen11
5 Replies

4. Shell Programming and Scripting

Print Only second Duplicate entry in the file

I have file where it contains 2 columns. In two columns the first column is repeated more than once. I wanted to take the unique record in first column and the corresponding second column value . The below is the example of the file: 8244100320012955|000b063471a4... (4 Replies)
Discussion started by: ravi_rn
4 Replies

5. Shell Programming and Scripting

Deleting file entry

Hello everyone, I want to compare the first line of a file(ABC) with that of a folder,XYZ(folder contents) and want that line to be deleted from the file(ABC) if that entry doesn't exist in the folder(XYZ) I want to put this in a loop. please can anyone help thanks (6 Replies)
Discussion started by: swasid
6 Replies

6. Shell Programming and Scripting

Need to delete duplicate lease entry

Hi *, I need to delete duplicate lease entries in file according to MAC/IP. I'm having tempfile which contains many lease info and need to have one entry for each IP(not more than that), if it contains more than one entry for same set, need to be deleted that entry... EX: lease... (4 Replies)
Discussion started by: SMNK
4 Replies

7. Red Hat

Adding or deleting an entry in /etc/inittab without using vi editrors or any editor.

Hi masters Is there any way to edit or delete an entry in inittab file without using vi or any editors? We can use commands instead or any shell script .. If any one can help deeply appreciated Thanks a lot sai (3 Replies)
Discussion started by: saidiya
3 Replies

8. Shell Programming and Scripting

Deleting Duplicates leaving the first entry

Hi, I need to delete duplicate records in a file that is around 30MB. Below is what I need. Below are the entries of input file and the output file that I need. Each section of input file is separated by an empty line. Some of these sections have duplicate uid values. I want to retain only one... (4 Replies)
Discussion started by: Samingla
4 Replies

9. Shell Programming and Scripting

Searching for an entry and deleting the line

Hi Im trying to scan a file for certain entries and remove their corresponding lines completely. What I have now is this, for USER in user1 user2 user3 user4 do sed '/$USER/d' /etc/sudoers done However this doesn't remove the entries at all. Is there another way for this? Thanks... (2 Replies)
Discussion started by: bludhemn
2 Replies

10. Shell Programming and Scripting

checking duplicate entry in file

Hi i have a file like 110.10 120.10 -1120 110.10 and the lines are having more than 10k. do we have anycommand to check the duplicate entries in the file. I applied the while loop by greping each line with whole file, but it is taking huge amount of time as the file size is large. ... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

LEARN ABOUT SUSE

pfunc

PFUNC(1)						User Contributed Perl Documentation						  PFUNC(1)

NAME

       pfunc - grep for perl functions

SYNOPSIS

	   pfunc subroutine FILES...

DESCRIPTION

       pfunc searches the named FILES for all calls to the given subroutine.  It will report back the file and line number each call is found on
       along with what sort of call it is

	   function	       foo()
	   class method        Class->foo()
	   object method       $obj->foo()

EXAMPLE

	   $ pfunc isa /usr/share/perl/5.6.1/*.pm
	   Called as function in /usr/share/perl/5.6.1/CGI.pm at line 316
	   Called as function in /usr/share/perl/5.6.1/CGI.pm at line 327
	   Called as function in /usr/share/perl/5.6.1/CGI.pm at line 397
	   Called as function in /usr/share/perl/5.6.1/CGI.pm at line 494
	   Called as function in /usr/share/perl/5.6.1/CGI.pm at line 495
	   Called as object method in /usr/share/perl/5.6.1/CPAN.pm at line 4957
	   Called as function in /usr/share/perl/5.6.1/Dumpvalue.pm at line 191
	   Called as function in /usr/share/perl/5.6.1/Dumpvalue.pm at line 218
	   Called as function in /usr/share/perl/5.6.1/Dumpvalue.pm at line 248
	   Called as function in /usr/share/perl/5.6.1/Dumpvalue.pm at line 251
	   Called as function in /usr/share/perl/5.6.1/Dumpvalue.pm at line 254
	   Called as object method in /usr/share/perl/5.6.1/Shell.pm at line 28
	   Called as object method in /usr/share/perl/5.6.1/base.pm at line 12

NOTES

       Its not fast, but its accurate.

AUTHOR

       Michael G Schwern <schwern@pobox.com>

SEE ALSO

       Module::Info

perl v5.12.1							    2002-12-05								  PFUNC(1)
All times are GMT -4. The time now is 08:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy