|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Extract and count number of Duplicate rows
Post 302777585 by Arun Mishra on Friday 8th of March 2013 06:48:34 AM
03-08-2013
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a input file with formating:
6000000901 ;36200103 ;h3a01f496 ;
2000123605 ;36218982 ;heefa1328 ;
2000273132 ;36246985 ;h08c5cb71 ;
2000041207 ;36246985 ;heef75497 ;
Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies
2. Shell Programming and Scripting
I have searched the internet for duplicate row extracting.
All I have seen is extracting good rows or eliminating duplicate rows.
How do I extract duplicate rows from a flat file in unix.
I'm using Korn shell on HP Unix.
For.eg.
FlatFile.txt
========
123:456:678
123:456:678
123:456:876... (5 Replies)
Discussion started by: bobbygsk
5 Replies
3. UNIX for Dummies Questions & Answers
Hi All,
I have the following input which i want to process using AWK.
Rows,NC,amount
1,1202,0.192387
2,1201,0.111111
3,1201,0.123456
i want the following output
count of rows = 3 ,sum of amount = 0.426954
Many thanks (2 Replies)
Discussion started by: pistachio
2 Replies
4. Shell Programming and Scripting
Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count
The input file:
21 2341 A
21 2341 A
21 2341 A
21 2341 C
21 2341 C
21 2341 C
21 2341 C
21 4567 A
21 4567 A
21 4567 C
... (6 Replies)
Discussion started by: juelillo
6 Replies
5. UNIX for Dummies Questions & Answers
Could anybody help with this?
I have input below .....
david,39
david,39
emelie,40
clarissa,22
bob,42
bob,42
tim,32
bob,39
david,38
emelie,47
what i want to do is count how many names there are with different ages, so output would be like this ....
david,2
emelie,2
clarissa,1... (3 Replies)
Discussion started by: itsme999
3 Replies
6. Shell Programming and Scripting
Hi! I have a file as below:
line1
line2
line2
line3
line3
line3
line4
line4
line4
line4
I would like to extract duplicate lines (not unique, triplicate or quadruplicate lines). Output will be as below:
line2
line2
I would appreciate if anyone can help. Thanks. (4 Replies)
Discussion started by: chromatin
4 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I need a solaris shell script to read multiple files and count number of unique name rows(strings) from those files. The input and output should be like this
Input:
file 1
abc
cde
abc ... (9 Replies)
Discussion started by: ssk250
9 Replies
8. Shell Programming and Scripting
Gents
Can you help please.
Input file
5490921425 1 7 1310342 54909214251
5490921425 2 1 1 54909214252
5491120937 1 1 3 54911209371
5491120937 3 1 1 54911209373
5491320785 1 ... (4 Replies)
Discussion started by: jiam912
4 Replies
9. Shell Programming and Scripting
Hello
I have a file like this:
> cat examplefile
ghi|NN603762|eee
mno|NN607265|ttt
pqr|NN613879|yyy
stu|NN615002|uuu
jkl|NN607265|rrr
vwx|NN615002|iii
yzA|NN618555|ooo
def|NN190486|www
BCD|NN628717|ppp
abc|NN190486|qqq
EFG|NN628717|aaa
HIJ|NN628717|sss
>
I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies
10. Shell Programming and Scripting
I have a file with 48 rows. I am counting 6 rows and adding 6 to that number and repeating the operation, and then output the value in column 1. For the second column, I would like to get sort of a binary output (1s and 2s) every 3rd row. This is what I have:
awk '{print ++src +... (1 Reply)
Discussion started by: Xterra
1 Replies
LEARN ABOUT MOJAVE
sys::hostname::long5.18
Sys::Hostname::Long(3) User Contributed Perl Documentation Sys::Hostname::Long(3) NAME
Sys::Hostname::Long - Try every conceivable way to get full hostname SYNOPSIS
use Sys::Hostname::Long; $host_long = hostname_long; DESCRIPTION
How to get the host full name in perl on multiple operating systems (mac, windows, unix* etc) DISCUSSION
This is the SECOND release of this code. It has an improved set of tests and improved interfaces - but it is still often failing to get a full host name. This of course is the reason I wrote the module, it is difficult to get full host names accurately on each system. On some systems (eg: Linux) it is dependent on the order of the entries in /etc/hosts. To make it easier to test I have testall.pl to generate an output list of all methods. Thus even if the logic is incorrect, it may be possible to get the full name. Attempt via many methods to get the systems full name. The Sys::Hostname class is the best and standard way to get the system hostname. However it is missing the long hostname. Special thanks to David Sundstrom and Greg Bacon for the original Sys::Hostname SUPPORT
This is the original list of platforms tested. MacOS Macintosh Classic OK Win32 MS Windows (95,98,nt,2000...) 98 OK MacOS X Macintosh 10 OK (other darwin) Probably OK (not tested) Linux Linux UNIX OS OK Sparc OK HPUX H.P. Unix 10? Not Tested Solaris SUN Solaris 7? OK (now) Irix SGI Irix 5? Not Tested FreeBSD FreeBSD OK A new list has now been compiled of all the operating systems so that I can individually keep informaiton on their success. THIS IS IN NEED OF AN UPDATE AFTER NEXT RELEASE. Acorn - Not yet tested AIX - Not yet tested Amiga - Not yet tested Atari - Not yet tested AtheOS - Not yet tested BeOS - Not yet tested BSD - Not yet tested BSD/OS - Not yet tested Compaq - Not yet tested Cygwin - Not yet tested Concurrent - Not yet tested DG/UX - Not yet tested Digital - Not yet tested DEC OSF/1 - Not yet tested Digital UNIX - Not yet tested DYNIX/ptx - Not yet tested EPOC - Not yet tested FreeBSD - Not yet tested Fujitsu-Siemens - Not yet tested Guardian - Not yet tested HP - Not yet tested HP-UX - Not yet tested IBM - Not yet tested IRIX - Not yet tested - 3rd hand information might be ok. Japanese - Not yet tested JPerl - Not yet tested Linux Debian - Not yet tested Gentoo - Not yet tested Mandrake - Not yet tested Red Hat- Not yet tested Slackware - Not yet tested SuSe - Not yet tested Yellowdog - Not yet tested LynxOS - Not yet tested Mac OS - Not yet tested Mac OS X - OK 20040315 (v1.1) MachTen - Not yet tested Minix - Not yet tested MinGW - Not yet tested MiNT - Not yet tested MPE/iX - Not yet tested MS-DOS - Not yet tested MVS - Not yet tested NetBSD - Not yet tested NetWare - Not yet tested NEWS-OS - Not yet tested NextStep - Not yet tested Novell - Not yet tested NonStop - Not yet tested NonStop-UX - Not yet tested OpenBSD - Not yet tested ODT - Not yet tested OpenVMS - Not yet tested Open UNIX - Not yet tested OS/2 - Not yet tested OS/390 - Not yet tested OS/400 - Not yet tested OSF/1 - Not yet tested OSR - Not yet tested Plan 9 - Not yet tested Pocket PC - Not yet tested PowerMAX - Not yet tested Psion - Not yet tested QNX 4 - Not yet tested 6 (Neutrino) - Not yet tested Reliant UNIX - Not yet tested RISCOS - Not yet tested SCO - Not yet tested SGI - Not yet tested Symbian - Not yet tested Sequent - Not yet tested Siemens - Not yet tested SINIX - Not yet tested Solaris - Not yet tested SONY - Not yet tested Sun - Not yet tested Stratus - Not yet tested Tandem - Not yet tested Tru64 - Not yet tested Ultrix - Not yet tested UNIX - Not yet tested U/WIN - Not yet tested Unixware - Not yet tested VMS - Not yet tested VOS - Not yet tested Windows CE - Not yet tested 3.1 - Not yet tested 95 - Not yet tested 98 - Not yet tested Me - Not yet tested NT - Not yet tested 2000 - Not yet tested XP - Not yet tested z/OS - Not yet tested KNOWN LIMITATIONS
Unix Most unix systems have trouble working out the fully quallified domain name as it to be configured somewhere in the system correctly. For example in most linux systems (debian, ?) the fully qualified name should be the first entry next to the ip number in /etc/hosts 192.168.0.1 fred.somwhere.special fred If it is the other way around, it will fail. Mac TODO
Contributions David Dick Graeme Hart Piotr Klaban * Extra code from G * Dispatch table * List of all operating systems. Solaris * Fall back 2 - TCP with DNS works ok * Also can read /etc/defaultdomain file SEE ALSO
L<Sys::Hostname> AUTHOR
Scott Penrose <scottp@dd.com.au> COPYRIGHT
Copyright (c) 2001,2004,2005 Scott Penrose. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.18.2 2005-06-25 Sys::Hostname::Long(3)