9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all,
I have huge a tab-delimited file with the following format and I want to remove the duplicates according to their frequency based on Column2 and Column3.
Column1 Column2 Column3 Column4 Column5 Column6 Column7
1 user1 access1 word word 3 2
2 user2 access2 ... (10 Replies)
Discussion started by: corfuitl
10 Replies
2. Shell Programming and Scripting
I am trying to see if I can use awk to remove duplicates from a file. This is the file:
-==> Listvol <==
deleting /vol/eng_rmd_0941
deleting /vol/eng_rmd_0943
deleting /vol/eng_rmd_0943
deleting /vol/eng_rmd_1006
deleting /vol/eng_rmd_1012
rearrange /vol/eng_rmd_0943
... (6 Replies)
Discussion started by: newbie2010
6 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I have a tablular separated file and I want to remove all the rows that have duplicates. The diuplicates I need to check are in column 13.
I have tried to use awk but I have no Idea how to keep the duplicate file.
awk 'FNR==NR{a++;next}(a> 1)' tomodify.txt tomodify.txt > new.txt
... (4 Replies)
Discussion started by: flacchy
4 Replies
4. Shell Programming and Scripting
Hi,
I have the following file,
chr1 100 200 20
chr1 201 300 22
chr1 220 345 23
chr1 230 456 33.5
chr1 243 567 90
chr1 345 600 20
chr1 430 619 21.78
chr1 870 910 112.3
chr1 914 920 12
chr1 930 999 13
My output would be
peak1 20 22 23 33.5 90
peak2 20 21.78 112.3 12 13
Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies
5. Shell Programming and Scripting
Hi all,
I have an input file like this
Now
I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be
Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies
6. Shell Programming and Scripting
Hi,
I have a requirement where I have to remove duplicates from a file based on the first 8 chars (It is fixed width file of 10 chars length) and whenever a duplicate row is found, its original row's last 2 chars should be updated to all 0's.
I thought of using
sort -u -k 1.1,1.8... (4 Replies)
Discussion started by: farawaydsky
4 Replies
7. Shell Programming and Scripting
Hello,
I am new to shell scripting. I have a huge file with multiple columns for example:
I have 5 columns below.
HWUSI-EAS000_29:1:105 + chr5 76654650 AATTGGAA HHHHG
HWUSI-EAS000_29:1:106 + chr5 76654650 AATTGGAA B@HYL
HWUSI-EAS000_29:1:108 + ... (4 Replies)
Discussion started by: Diya123
4 Replies
8. Shell Programming and Scripting
Given a file such as this I need to remove the duplicates.
00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt
00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt
0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt
0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies
9. UNIX for Dummies Questions & Answers
Hi,
How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters.
The file is a fixed width file.
ex of Record:
DTYU12333567opert tjhi kkklTRG9012
The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies
