Sponsored Content
Top Forums Shell Programming and Scripting Help with replace duplicate content Post 302584039 by cpp_beginner on Thursday 22nd of December 2011 03:15:20 AM
Old 12-22-2011
Help with replace duplicate content

Input file:
Code:
CCNI	data564_input1	264
CORO1A	data564_input2	155
ABC-B	data17_input1	3466
ABC-B	data17_input2	1133
ABC-B	data17_input3	2162
ABC-B	data17_input4	2019
HNRNPA2B1	data95_input1	101
HNRNPA2B1	data95_input2	340
IFITM1	data105_input2	291
IFITM2	data105_input1	505
MYL12A	data352_input2	212
MYL12B	data352_input1	131
MYL12B	data352_input3	76

Desired output file:
Code:
CCNI	data564_input1	264
CORO1A	data564_input2	155
ABC-B	data17_input1	3466
	data17_input2	1133
	data17_input3	2162
	data17_input4	2019
HNRNPA2B1	data95_input1	101
		data95_input2	340
IFITM1	data105_input2	291
IFITM2	data105_input1	505
MYL12A	data352_input2	212
MYL12B	data352_input1	131
	data352_input3	76

A tab delimiter "\t" is located in between each column.
I would like to replace the those duplicate content in column 1 with empty.
Thanks for any advice.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with remove duplicate content and only keep the first content detail

Input data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_21 SSA data_19 TYUEC data_14 TYUE data_15 SSA data_32 PEOCV . . Desired Output data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_19 TYUEC (9 Replies)
Discussion started by: patrick87
9 Replies

2. Shell Programming and Scripting

Help with remove duplicate content

Input file data_1 10 US data_1 2 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA data_3 4 CHINA . . data_60 123 US data_60 23 UK data_60 45 US Desired output file data_1 10 US data_1 5 UK data_2 20 ENGLAND data_2 12 KOREA (2 Replies)
Discussion started by: perl_beginner
2 Replies

3. Shell Programming and Scripting

Search duplicate field and replace one of them with new value

Dear All, I have file with 4 columns: 1 AA 0 21 2 BB 0 31 3 AA 0 21 4 CC 0 41 I would like to find the duplicate record based on column 2 and replace the 4th column of the duplicate by a new value. So, the output will be: 1 AA 0 21 2 BB 0 31 3 AA 0 -21 4 CC 0 41 Any suggestions... (3 Replies)
Discussion started by: ezhil01
3 Replies

4. Shell Programming and Scripting

Help with duplicate data content problem asking

Input file: A_69510335_ASD>aw 1199470 USA A_119571157_C>awe,QWEQE 113932840 USA C_34646666_qwe>TAWTT,G,TT 112736796 UK C_69510335_QW>T 1199470 USA D_70520237_WR>QEE,G 34459863 UK D_71380003_QWR>T 145418226 IK . Desired output: A_69510335_ASD>aw 1199470 USA... (1 Reply)
Discussion started by: perl_beginner
1 Replies

5. Shell Programming and Scripting

Replace duplicate columns with values from first occurrence

I've a text file with below values viz. multiple rows with same values in column 3, 4 and 5, which need to be considered as duplicates. For all such cases, the rows from second occurrence onwards should be modified in a way that their values in first two columns are replaced with values as in first... (4 Replies)
Discussion started by: asyed
4 Replies

6. Shell Programming and Scripting

Help with duplicate common data content

Input file: #data_131 0 >content..._* 1 >content..._at_+/97.20% #data_137 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3 >content..._at_+/97.20% 4 >content..._at_+/97.20% #data_141 0 >content..._* #data_150 0 >content..._* 1 >content..._at_+/97.20% 2 >seq..._* 3... (3 Replies)
Discussion started by: perl_beginner
3 Replies

7. Shell Programming and Scripting

Sed: replace content from file with the content from file

Hi, I am having trouble while using 'sed' with reading files. Please help. I have 3 files. File A, file B and file C. I want to find content of file B in file A and replace it by content in file C. Thanks a lot!! Here is a sample of my question. e.g. (file A: a.txt; file B: b.txt; file... (3 Replies)
Discussion started by: dirkaulo
3 Replies

8. Shell Programming and Scripting

Remove the duplicate content in a file

Here is the contents of test.txt Dependencies Resolved Changes in packages about to be updated: ChangeLog for: 1:perl-Archive-Extract-0.38-131.el6_4.x86_64, - Resolves: #915692 - CVE-2013-1667 (DoS in rehashing code) Dependencies Resolved Changes in packages about to be updated: ... (5 Replies)
Discussion started by: ashokvpp
5 Replies

9. Shell Programming and Scripting

Help with replace all the content within ()

Hi, Below is my input file : AAAG(12) TC(14) AACCCT(66) AACCCT(30) AACCCT(18) AACCCT(48) TCTG(12) TCTG(20) TCTG(16) AC(12) AC(12) TCTG(16) TCTG(12) AC(12) AC(12) AC(12) AC(26) AC(14) AGTG(12) AC(24) AGTG(12) TCC(12) Desired output : AAAG TC AACCCT AACCCT AACCCT AACCCT TCTG TCTG... (4 Replies)
Discussion started by: perl_beginner
4 Replies

10. Shell Programming and Scripting

Replace Content

Hello all ; ) I'got a file1 with a lot of emails like : fistname.lastname@domaine1.comAnd another file2 with emails like fistname.lastname@domaine2.ct.netI need a shell script that will read each line from the file1 and try to find if in file2 the fistname.lastname exist. If yes, the... (1 Reply)
Discussion started by: Aswex
1 Replies
geniconvtbl(1)							   User Commands						    geniconvtbl(1)

NAME
geniconvtbl - generate iconv code conversion tables SYNOPSIS
geniconvtbl [-fnq] [-p preprocessor] [-W arg] [-Dname] [-Dname=def] [-Idirectory] [-Uname] [infile...] DESCRIPTION
The geniconvtbl utility accepts code conversion rules defined in flat text file(s) and writes code conversion binary table file(s) that can be used to support user-defined iconv code conversions (see iconv(1) and iconv(3C) for more detail on the iconv code conversion). OPTIONS
The following options are supported: -f Overwrites output file if the output file exists. -n Does not generate an output file. This is useful to check the contents of the input file. -p preprocessor Uses specified preprocessor instead of the default preprocessor, /usr/lib/cpp. -q Quiet option. It suppresses warning and error messages. -W arg Passes the argument arg to the preprocessor. If this option is specified more than once, all arguments are passed to the preprocessor. -Dname geniconvtbl recognizes these options and passes them and their arguments to the preprocessor. -Dname=def -Idirectory -Uname OPERANDS
The following operand is supported: infile A path name of an input file. If no input file is specified, geniconvtbl reads from the standard input stream. The user can specify more than one input file if necessary. OUTPUT
If input is from the standard input stream, geniconvtbl writes output to the standard output stream. If one or more input files are speci- fied, geniconvtbl reads from each input file and writes to a corresponding output file. Each of the output file names will be the same as the corresponding input file with .bt appended. The generated output files must be moved to the following directory prior to using the code conversions at iconv(1) and iconv(3C): /usr/lib/iconv/geniconvtbl/binarytables/ The output file name should start with one or more printable ASCII characters as the 'fromcode' name followed by a percentage character (%), followed by one or more printable ASCII characters as the 'tocode' name, followed by the suffix '.bt'. The 'fromcode' and 'tocode' names are used to identify the iconv code conversion at iconv(1) and iconv_open(3C)). The properly named output file should be placed in the directory, /usr/lib/iconv/geniconvtbl/binarytables/. EXAMPLES
Example 1: Generating an iconv code conversion binary table The following example generates a code conversion binary table with output file name convertA2B.bt: example% geniconvtbl convertA2B Example 2: Generating multiple iconv code conversion binary tables The following example generates two code conversion binary tables with output files test1.bt and test2.bt: example% geniconvtbl test1 test2 Example 3: Using another preprocessor The following example generates a code conversion binary table once the specified preprocessor has processed the input file: example% geniconvtbl -p /opt/SUNWspro/bin/cc -W -E convertB2A Example 4: Placing a binary table To use the binary table created in the first example above as the engine of the conversion 'fromcode' ABC to 'tocode' DEF, become super- user and then rename it and place it like this: example# mv convertA2B.bt /usr/lib/iconv/geniconvtbl/binarytables/ABC%DEF.bt Example 5: Providing modified ISO8859-1 to UTF-8 code conversion Write a geniconvtbl source file that defines the code conversion. For instance, you can copy over /usr/lib/iconv/genicon- vtbl/srcs/ISO8859-1_to_UTF-8.src into your directory and make necessary changes at the source file. Once the modifications are done, generate the binary table: example% geniconvtbl ISO8859-1_to_UTF-8.src As super-user, place the generated binary table with a unique name at the system directory where iconv_open(3C) can find the binary table: example su Password: example% cp ISO8859-1_to_UTF-8.bt /usr/lib/iconv/geniconvtbl/binarytables/my-iso-8859-1%utf-8.bt After that, you can do the iconv code conversion. For instance: example% iconv -f my-iso-8859-1 -t utf-8 testfile.txt ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of geniconvtbl: LANG and LC_CTYPE. EXIT STATUS
The following exit values are returned: 0 No errors occurred and the output files were successfully created. 1 Command line options are not correctly used or an unknown command line option was specified. 2 Invalid input or output file was specified. 3 Conversion rules in input files are not correctly defined. 4 Conversion rule limit of input files has been reached. See NOTES section of geniconvtbl(4). 5 No more system resource error. 6 Internal error. FILES
/usr/lib/iconv/geniconvtbl/binarytables/*.bt conversion binary tables /usr/lib/iconv/geniconvtbl/srcs/* conversion source files for user reference ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
cpp(1), iconv(1), iconv(3C), iconv_close(3C), iconv_open(3C), geniconvtbl(4), attributes(5), environ(5), iconv(5) Solaris Internationalization Guide for Developers NOTES
The generated and correctly placed output files, /usr/lib/iconv/geniconvtbl/binarytables/*.bt, are used in both 32-bit and 64-bit environ- ments. SunOS 5.10 30 Nov 2001 geniconvtbl(1)
All times are GMT -4. The time now is 02:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy