Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Kindly check:remove duplicates with similar data in front of it
Top Forums Shell Programming and Scripting Kindly check:remove duplicates with similar data in front of it Post 302678525 by rangarasan on Saturday 28th of July 2012 07:26:08 AM
Old 07-28-2012
awk
Hi,
The input file has some pattern match related characters like []. I have not tested the below code. Make a try with this.
Code:
awk '{$0=gensub(/([\]\[\(\)\{\}])/,"\\\1","g",$0);t=$0;r=$1"";sub(r,"",t);if(a[$1]!~t){a[$1]=a[$1]""t;}else{if(!a[$1]){a[$1]=t;}}}END{for(i in a){print i,a[i];}}' file1

you have to escape the special characters before going to use those in regex. You can also use quotemeta function in perl and then pass those output lines to awk.
Cheers,
Ranga:-)
Last edited by rangarasan; 07-28-2012 at 08:40 AM.. Reason: add perl func name
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove space in front or end of each field

Hi, I have a txt file called a.txt which contain over 10,000 records and I would like to remove space before comma or after comma....like below: The input (for example two record 00001,00002): 00001,client,card limited ,02292,N ,162:41 , 192, ... (6 Replies)
Discussion started by: happyv
6 Replies

2. Shell Programming and Scripting

sh, ksh: command to remove front spaces from a string?

dear pro-coders, is there any command out there that takes out the front spaces from a string? sample strings: 4 members 5 members 3 members but it has to be like so: 4 members 5 members 3 members (3 Replies)
Discussion started by: pseudocoder
3 Replies

3. Shell Programming and Scripting

Request to check:Remove duplicates

Hi all I have a file with following kind of data I want to remove duplicates according to first column so that output contains Kindly let me scripting regading this. (4 Replies)
Discussion started by: manigrover
4 Replies

4. Shell Programming and Scripting

Request to check remove duplicates but write before it

Hi alll I have a file with following kind input I want in output duplicates should not be there but there should be numbering mentioned before that like (4 Replies)
Discussion started by: manigrover
4 Replies

5. Shell Programming and Scripting

Request to check:remove duplicates and write sytematically

Hi all I have a file with following input It contains 5 columns gene name drug drug ID disease approved Now the same gene is repeated many times with different data in column2,3 ,4,5 I want to arrange dat in such a way that there shuld be one entry in the column(no... (2 Replies)
Discussion started by: manigrover
2 Replies

6. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

7. Shell Programming and Scripting

Kindly check it: Camparison of files only column1 of 2 files

Hi all, I have 2 files in which i have to find commom entries in column 1 an dif soemthing is common write other data of both files in front of it mentioned. Gene symbol and disease name column 1 column2 ARFGEF2 CAD DDEF2 CAD PSCD3 CAD PSCD4 CAD CAMK1... (15 Replies)
Discussion started by: manigrover
15 Replies

8. Shell Programming and Scripting

common entries of first column in 2 or 3 files:kindly check

Hi all, I have 3 files with such data first files second file third file I have to find common entries of first column in two ways 1) between 2 files (2 Replies)
Discussion started by: manigrover
2 Replies

9. Shell Programming and Scripting

Sort data by date first and then remove duplicates

Hi , I have below data inside a file named ref.psv . I want to create a shell script which will do the below 2 points : (1) sort the file content first based on the latest date which is the last column in the file (actual file its the 175th column) (2)after sorting the file based on latest date... (3 Replies)
Discussion started by: samrat dutta
3 Replies

LEARN ABOUT DEBIAN

update-metainit

UPDATE-METAINIT(1)					User Contributed Perl Documentation					UPDATE-METAINIT(1)

NAME

       update-metainit - Generates init scripts

SYNOPSIS

       update-metainit [--remove initname [--purge] ]

DESCRIPTION

       Metainit solves the problem of writing good init scripts. Instead of manually creating these important files, they are derived from a
       declaritive description in the metainit files in /etc/metainit. These files can be shipped with packages or created by the local adminis-
       trator.

       If update-metainit called without argument, it will regenerate init scripts for all the files in /etc/metainit. The generated files contain
       a large warning in form of a comment that they will be overridden. Modifications are preferably done in the files in /etc/metainit and made
       effective by running update-metainit. If needed, the administrator can prevent modified init files by removing the warning comment.

OPTIONS

       --remove initname
	   This command will remove any generated and non-modified scripts that were created by the metainit file with the name initname.

       --purge
	   Only usable with --remove. Will remove the generated files even if modified.

SEE ALSO

       dh_metainit(1)

AUTHOR

       Joachim Breitner <nomeata@debian.org>

perl v5.8.8							    2007-07-30							UPDATE-METAINIT(1)
All times are GMT -4. The time now is 05:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy