Sponsored Content
Top Forums Shell Programming and Scripting Extract and count number of Duplicate rows Post 302777817 by franzpizzo on Friday 8th of March 2013 12:14:54 PM
Old 03-08-2013
Code:
awk '
{s[$0]++}              # this populate an array, the number of elements is the distinct value in the file (A B C) 
END {                  # and the value is the count of each element: eg. if i=A --> s[i]=3
  for(i in s) {        # for each distinct value i in s
  for(j=1;j<s[i];j++){ # s[i] is the count of element i: in this way
      print i;         # print s[i]-1 times the element i
  }
  }
}' ${TMP_DUPE_RECS}>${TMP_BAD_DATA_DUPE_RECS}

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract duplicate fields in rows

I have a input file with formating: 6000000901 ;36200103 ;h3a01f496 ; 2000123605 ;36218982 ;heefa1328 ; 2000273132 ;36246985 ;h08c5cb71 ; 2000041207 ;36246985 ;heef75497 ; Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies

2. Shell Programming and Scripting

How to extract duplicate rows

I have searched the internet for duplicate row extracting. All I have seen is extracting good rows or eliminating duplicate rows. How do I extract duplicate rows from a flat file in unix. I'm using Korn shell on HP Unix. For.eg. FlatFile.txt ======== 123:456:678 123:456:678 123:456:876... (5 Replies)
Discussion started by: bobbygsk
5 Replies

3. UNIX for Dummies Questions & Answers

how to count number of rows and sum of column using awk

Hi All, I have the following input which i want to process using AWK. Rows,NC,amount 1,1202,0.192387 2,1201,0.111111 3,1201,0.123456 i want the following output count of rows = 3 ,sum of amount = 0.426954 Many thanks (2 Replies)
Discussion started by: pistachio
2 Replies

4. Shell Programming and Scripting

how to add the number of row and count number of rows

Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count The input file: 21 2341 A 21 2341 A 21 2341 A 21 2341 C 21 2341 C 21 2341 C 21 2341 C 21 4567 A 21 4567 A 21 4567 C ... (6 Replies)
Discussion started by: juelillo
6 Replies

5. UNIX for Dummies Questions & Answers

count number of rows based on other column values

Could anybody help with this? I have input below ..... david,39 david,39 emelie,40 clarissa,22 bob,42 bob,42 tim,32 bob,39 david,38 emelie,47 what i want to do is count how many names there are with different ages, so output would be like this .... david,2 emelie,2 clarissa,1... (3 Replies)
Discussion started by: itsme999
3 Replies

6. Shell Programming and Scripting

How to extract duplicate rows

Hi! I have a file as below: line1 line2 line2 line3 line3 line3 line4 line4 line4 line4 I would like to extract duplicate lines (not unique, triplicate or quadruplicate lines). Output will be as below: line2 line2 I would appreciate if anyone can help. Thanks. (4 Replies)
Discussion started by: chromatin
4 Replies

7. UNIX for Dummies Questions & Answers

Script to count number of rows

Hi, I need a solaris shell script to read multiple files and count number of unique name rows(strings) from those files. The input and output should be like this Input: file 1 abc cde abc ... (9 Replies)
Discussion started by: ssk250
9 Replies

8. Shell Programming and Scripting

Extract duplicate rows with conditions

Gents Can you help please. Input file 5490921425 1 7 1310342 54909214251 5490921425 2 1 1 54909214252 5491120937 1 1 3 54911209371 5491120937 3 1 1 54911209373 5491320785 1 ... (4 Replies)
Discussion started by: jiam912
4 Replies

9. Shell Programming and Scripting

Extract and exclude rows based on duplicate values

Hello I have a file like this: > cat examplefile ghi|NN603762|eee mno|NN607265|ttt pqr|NN613879|yyy stu|NN615002|uuu jkl|NN607265|rrr vwx|NN615002|iii yzA|NN618555|ooo def|NN190486|www BCD|NN628717|ppp abc|NN190486|qqq EFG|NN628717|aaa HIJ|NN628717|sss > I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies

10. Shell Programming and Scripting

Reseting row count every given number of rows

I have a file with 48 rows. I am counting 6 rows and adding 6 to that number and repeating the operation, and then output the value in column 1. For the second column, I would like to get sort of a binary output (1s and 2s) every 3rd row. This is what I have: awk '{print ++src +... (1 Reply)
Discussion started by: Xterra
1 Replies
MYSQL_AFFECTED_ROWS(3)							 1						    MYSQL_AFFECTED_ROWS(3)

mysql_affected_rows - Get number of affected rows in previous MySQL operation

SYNOPSIS
Warning This extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQL extension should be used. See also MySQL: choosing an API guide and related FAQ for more information. Alternatives to this function include: omysqli_affected_rows(3) o PDOStatement::rowCount int mysql_affected_rows ([resource $link_identifier = NULL]) DESCRIPTION
Get the number of affected rows by the last INSERT, UPDATE, REPLACE or DELETE query associated with $link_identifier. o $ link_identifier -The MySQL connection. If the link identifier is not specified, the last link opened by mysql_connect(3) is assumed. If no such link is found, it will try to create one as if mysql_connect(3) was called with no arguments. If no connection is found or established, an E_WARNING level error is generated. Returns the number of affected rows on success, and -1 if the last query failed. If the last query was a DELETE query with no WHERE clause, all of the records will have been deleted from the table but this function will return zero with MySQL versions prior to 4.1.2. When using UPDATE, MySQL will not update columns where the new value is the same as the old value. This creates the possibility that mysql_affected_rows(3) may not actually equal the number of rows matched, only the number of rows that were literally affected by the query. The REPLACE statement first deletes the record with the same primary key and then inserts the new record. This function returns the number of deleted records plus the number of inserted records. In the case of "INSERT ... ON DUPLICATE KEY UPDATE" queries, the return value will be 1 if an insert was performed, or 2 for an update of an existing row. Example #1 mysql_affected_rows(3) example <?php $link = mysql_connect('localhost', 'mysql_user', 'mysql_password'); if (!$link) { die('Could not connect: ' . mysql_error()); } mysql_select_db('mydb'); /* this should return the correct numbers of deleted records */ mysql_query('DELETE FROM mytable WHERE id < 10'); printf("Records deleted: %d ", mysql_affected_rows()); /* with a where clause that is never true, it should return 0 */ mysql_query('DELETE FROM mytable WHERE 0'); printf("Records deleted: %d ", mysql_affected_rows()); ?> The above example will output something similar to: Records deleted: 10 Records deleted: 0 Example #2 mysql_affected_rows(3) example using transactions <?php $link = mysql_connect('localhost', 'mysql_user', 'mysql_password'); if (!$link) { die('Could not connect: ' . mysql_error()); } mysql_select_db('mydb'); /* Update records */ mysql_query("UPDATE mytable SET used=1 WHERE id < 10"); printf ("Updated records: %d ", mysql_affected_rows()); mysql_query("COMMIT"); ?> The above example will output something similar to: Updated Records: 10 Note Transactions If you are using transactions, you need to call mysql_affected_rows(3) after your INSERT, UPDATE, or DELETE query, not after the COMMIT. Note SELECT Statements To retrieve the number of rows returned by a SELECT, it is possible to use mysql_num_rows(3). Note Cascaded Foreign Keys mysql_affected_rows(3) does not count rows affected implicitly through the use of ON DELETE CASCADE and/or ON UPDATE CASCADE in for- eign key constraints. mysql_num_rows(3), mysql_info(3). PHP Documentation Group MYSQL_AFFECTED_ROWS(3)
All times are GMT -4. The time now is 08:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy