Sponsored Content
Top Forums Shell Programming and Scripting Count the repetition of a Field in File Post 302344565 by indian.ace on Monday 17th of August 2009 04:40:48 AM
Old 08-17-2009
Quote:
Originally Posted by tukuyomi
Here is what I get so far. Of course, you'll surely have other replies that will do the same in a simpler way Smilie
Code:
#!/bin/sh

sort -t',' -k2,2n file | uniq -c -s4 > tmp

while read line; do
  echo "$line,$(grep ${line##*,} tmp | awk '{print $1}')"
done < file

exit 0

Your data file need to be named file, in the same directory as the script.
I use a tmp file to keep the number of occurences of the second field.
---------- Post updated at 03:31 AM ---------- Previous update was at 03:29 AM ----------

[/COLOR]Hi Tukuyomi
Thanks for the solution but it has a deviation than expected result, and eating out some inputs. The output was like this.
1 pqp,11224
3 aaa,12345
1 stu,34567
1 pqr,51223
2 mno,72121
2 jkl,88888

can you please amend it if possible.Smilie

---------- Post updated at 03:40 AM ---------- Previous update was at 03:31 AM ----------

Hi frank,
there is no output for this awk script, its just publishing the same optput as input except a field saparator at the end as ",". Please can you correct it.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Count of Field for Non-Empty

Hi Guys, I wanted to count the number of records for a particular field of a file. whose fields are separated by comma"," I fI use this command. cat "filename" cut -sd "," -f13 | wc -l This shows all the lines count including the blank values for the field number 13. I wanted to count... (2 Replies)
Discussion started by: Swapna173
2 Replies

2. Shell Programming and Scripting

Count field frequency in a '|' delimited file

I have a large file with fields delimited by '|', and I want to run some analysis on it. What I want to do is count how many times each field is populated, or list the frequency of population for each field. I am in a Sun OS environment. Thanks, - CB (3 Replies)
Discussion started by: ChicagoBlues
3 Replies

3. Shell Programming and Scripting

How to check the repetition values in a file using bourne shell

Hi all, I have a scenario, like consider a file abc.txt, inside abc.txt, the contents is value1 = aaa, value2 = bbb, value3 = ccc, value1 = ddd. In this situation i need to throw an error for the repeatation of keys like "value1" is repeating twice. how to handle this using bourne... (1 Reply)
Discussion started by: Nandagopal
1 Replies

4. Shell Programming and Scripting

Count number of occurences of a character in a field defined by the character in another field

Hello, I have a text file with n lines in the following format (9 column fields): Example: contig00012 149606 G C 49 68 60 18 c$cccccacccccccccc^c I need to count the number of lower-case and upper-case occurences in column 9, respectively, of the... (3 Replies)
Discussion started by: s052866
3 Replies

5. Shell Programming and Scripting

Read File and Display The Count of a particular field

Hi Mates, I require help in the following: I have the following file snmp.txt Wed Mar 2 16:02:39 SGT 2011 Class : mmTrapBladeS origin : 10.0.0.0 hostname : 10.0.0.2 msg : IBM Blade Alert: Calendar Index : 10.0.0.2-IBMBLADE Fri Mar 4 07:10:54 SGT 2011 Class : mmTrapBladeS... (2 Replies)
Discussion started by: dbashyam
2 Replies

6. Shell Programming and Scripting

Help with awk for selecting lines in a file avoiding repetition

Hello, I am using Awk in UBUNTU 12.04. I have a file as following with 48,432,354 lines and 4 fields. The file has this structure (There are repetitions of the first column in several lines) AB_14 S54 A G AB_14 S55 A A AB_14 S56 G G GO_15 S45 T A GO_15 S46 A A PT_16 S33 C C PT_16 ... (4 Replies)
Discussion started by: Homa
4 Replies

7. Shell Programming and Scripting

Count the field values in a file

Hi I have a file with contents like : 101,6789556897,0000795369 - seq - fmt_recs187] - avg_recs 101,4678354769,0000835783 - seq - fmt_recs98] - avg_recs 221,5679787008,0001344589 - seq - fmt_recs1283] - avg_recs I need to find the sum of the all the values (which are in bold). here... (6 Replies)
Discussion started by: rkrish
6 Replies

8. Shell Programming and Scripting

Count of unique lines in field 4

When I use the below awk to count the unique lines in $4 for the input it seems to work. The answer is 3 because $4 is only unique 3 times in all the entries. However, when I use the same on actual data I get 56,536 and I know the answer should be 56,548. My question is there a better way to... (8 Replies)
Discussion started by: cmccabe
8 Replies

9. Shell Programming and Scripting

How to count the field and add String?

Example i have 3 fields and i wanna add my input to the field after that (NF+1) SID|Fname|Lname 123123:adds:asdasdasd Result SID|Fname|Lname|Number 123123:adds:asdasdasd:123123 ---------- Post updated at 02:36 PM ---------- Previous update was at 02:23 PM ---------- Input is likes.... (3 Replies)
Discussion started by: vutung1991
3 Replies

10. UNIX for Beginners Questions & Answers

Awk: count unique elements in a field and sum their occurence across the entire file

Hi, Sure it's an easy one, but it drives me insane. input ("|" separated): 1|A,B,C,A 2|A,D,D 3|A,B,B I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1. I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies
SORT(1) 							   User Commands							   SORT(1)

NAME
sort - sort lines of text files SYNOPSIS
sort [OPTION]... [FILE]... sort [OPTION]... --files0-from=F DESCRIPTION
Write sorted concatenation of all FILE(s) to standard output. Mandatory arguments to long options are mandatory for short options too. Ordering options: -b, --ignore-leading-blanks ignore leading blanks -d, --dictionary-order consider only blanks and alphanumeric characters -f, --ignore-case fold lower case to upper case characters -g, --general-numeric-sort compare according to general numerical value -i, --ignore-nonprinting consider only printable characters -M, --month-sort compare (unknown) < `JAN' < ... < `DEC' -n, --numeric-sort compare according to string numerical value -R, --random-sort sort by random hash of keys --random-source=FILE get random bytes from FILE (default /dev/urandom) -r, --reverse reverse the result of comparisons --sort=WORD sort according to WORD: general-numeric -g, month -M, numeric -n, random -R, version -V -V, --version-sort sort by numeric version Other options: --batch-size=NMERGE merge at most NMERGE inputs at once; for more use temp files -c, --check, --check=diagnose-first check for sorted input; do not sort -C, --check=quiet, --check=silent like -c, but do not report first bad line --compress-program=PROG compress temporaries with PROG; decompress them with PROG -d --files0-from=F read input from the files specified by NUL-terminated names in file F; If F is - then read names from standard input -k, --key=POS1[,POS2] start a key at POS1 (origin 1), end it at POS2 (default end of line) -m, --merge merge already sorted files; do not sort -o, --output=FILE write result to FILE instead of standard output -s, --stable stabilize sort by disabling last-resort comparison -S, --buffer-size=SIZE use SIZE for main memory buffer -t, --field-separator=SEP use SEP instead of non-blank to blank transition -T, --temporary-directory=DIR use DIR for temporaries, not $TMPDIR or /tmp; multiple options specify multiple directories -u, --unique with -c, check for strict ordering; without -c, output only the first of an equal run -z, --zero-terminated end lines with 0 byte, not newline --help display this help and exit --version output version information and exit POS is F[.C][OPTS], where F is the field number and C the character position in the field; both are origin 1. If neither -t nor -b is in effect, characters in a field are counted from the beginning of the preceding whitespace. OPTS is one or more single-letter ordering options, which override global ordering options for that key. If no key is given, use the entire line as the key. SIZE may be followed by the following multiplicative suffixes: % 1% of memory, b 1, K 1024 (default), and so on for M, G, T, P, E, Z, Y. With no FILE, or when FILE is -, read standard input. *** WARNING *** The locale specified by the environment affects sort order. Set LC_ALL=C to get the traditional sort order that uses native byte values. AUTHOR
Written by Mike Haertel and Paul Eggert. REPORTING BUGS
Report sort bugs to bug-coreutils@gnu.org GNU coreutils home page: <http://www.gnu.org/software/coreutils/> General help using GNU software: <http://www.gnu.org/gethelp/> COPYRIGHT
Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
The full documentation for sort is maintained as a Texinfo manual. If the info and sort programs are properly installed at your site, the command info coreutils 'sort invocation' should give you access to the complete manual. GNU coreutils 7.1 July 2010 SORT(1)
All times are GMT -4. The time now is 07:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy