Sponsored Content
Top Forums Shell Programming and Scripting Shell script to count unique rows in a CSV Post 302566146 by durden_tyler on Wednesday 19th of October 2011 03:21:07 PM
Old 10-19-2011
Quote:
Originally Posted by Nani369
...
I have a CSV file of 30 columns separated by ,. I want to get a count of all unique rows written to a flat file. The CSV file is around 5000 rows
The first column is a time stamp and I need to exclude while counting unique
...
Code:
$
$
$ cat f35
[10/17/11 20:48:45:213 EDT] , String 1 , String 2 , String 3, .....,String 29
[10/17/11 20:48:46:223 EDT] , String 1 , String 2 , String 3, .....,String 29
[10/17/11 20:48:46:224 EDT] , String 1a, String 2a, String 3a,.....,String 29a
$
$ cut -d, -f2- f35 | sort -u | wc -l
2
$
$

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to count unique number of user loged in

Hi all, I am taking my first course in unix and having some problems. I would line to create a scrip that does the following: 1) will tell me the number of users logged in. If a user is logged more than once just count it only ones. 2) want to be able to display the number of users that... (1 Reply)
Discussion started by: elchalateco
1 Replies

2. Shell Programming and Scripting

Please do help: Perl Script to pull out rows from a CSV file

I have CSV file that contains data in the format as shown below: ABC, 67, 56, 67, 78, 89, 76, 55 PDR, 85, 83, 83, 72, 82, 89, 83 MPG, 86, 53, 54, 65, 23, 54, 75 .. .. .. .. I want to create a script that will pull out the rows from the above sheet and paste it into another CSV file.... (12 Replies)
Discussion started by: pankajusc
12 Replies

3. Shell Programming and Scripting

List unique values and count instances in .csv file

I need to take the second column of a .csv file and count the number of instances of each unique value in that same second column. I'd like the output to be value,count sorted by most instances. Thanks for any guidance! Data example: 317476,317756,0 816063,318861,0 313123,319091,0... (4 Replies)
Discussion started by: batcho
4 Replies

4. Shell Programming and Scripting

Delete unique rows - optimize script

Hi all, I have the following input - the unique row key is 1st column cat file.txt A response C request C response D request C request C response E request The desired output should be C request (7 Replies)
Discussion started by: varu0612
7 Replies

5. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

6. Shell Programming and Scripting

Script to compare count of two csv files

Hi Guys, I need to write a script to compare the count of two csv files each having 5 columns. Everyday a csv file is recived. Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Discussion started by: Vivekit82
3 Replies

7. Shell Programming and Scripting

Unique extraction of rows

I do have a tab delimited file of the following format: 431 kat1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 432 kat2 2 NA NA NA NA NA NA NA NA NA NA NA NA NA 433 KATe NA 3 NA NA 6 NA NA NA 10 11 NA NA NA NA 542 Kaed 2 NA NA NA NA NA NA NA NA NA NA NA NA NA 543 hkwuy NA NA NA NA 6 NA NA NA NA 11 NA NA... (11 Replies)
Discussion started by: Kanja
11 Replies

8. UNIX for Dummies Questions & Answers

Script to count number of rows

Hi, I need a solaris shell script to read multiple files and count number of unique name rows(strings) from those files. The input and output should be like this Input: file 1 abc cde abc ... (9 Replies)
Discussion started by: ssk250
9 Replies

9. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

10. Shell Programming and Scripting

Finding total distinct count from multiple csv files through UNIX script

Hi All , I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct count on all files. We can't merge all the files here as file size are huge in millions.I have tried in below way for each... (9 Replies)
Discussion started by: STCET22
9 Replies
slags2.f(3)							      LAPACK							       slags2.f(3)

NAME
slags2.f - SYNOPSIS
Functions/Subroutines subroutine slags2 (UPPER, A1, A2, A3, B1, B2, B3, CSU, SNU, CSV, SNV, CSQ, SNQ) SLAGS2 computes 2-by-2 orthogonal matrices U, V, and Q, and applies them to matrices A and B such that the rows of the transformed A and B are parallel. Function/Subroutine Documentation subroutine slags2 (logicalUPPER, realA1, realA2, realA3, realB1, realB2, realB3, realCSU, realSNU, realCSV, realSNV, realCSQ, realSNQ) SLAGS2 computes 2-by-2 orthogonal matrices U, V, and Q, and applies them to matrices A and B such that the rows of the transformed A and B are parallel. Purpose: SLAGS2 computes 2-by-2 orthogonal matrices U, V and Q, such that if ( UPPER ) then U**T *A*Q = U**T *( A1 A2 )*Q = ( x 0 ) ( 0 A3 ) ( x x ) and V**T*B*Q = V**T *( B1 B2 )*Q = ( x 0 ) ( 0 B3 ) ( x x ) or if ( .NOT.UPPER ) then U**T *A*Q = U**T *( A1 0 )*Q = ( x x ) ( A2 A3 ) ( 0 x ) and V**T*B*Q = V**T*( B1 0 )*Q = ( x x ) ( B2 B3 ) ( 0 x ) The rows of the transformed A and B are parallel, where U = ( CSU SNU ), V = ( CSV SNV ), Q = ( CSQ SNQ ) ( -SNU CSU ) ( -SNV CSV ) ( -SNQ CSQ ) Z**T denotes the transpose of Z. Parameters: UPPER UPPER is LOGICAL = .TRUE.: the input matrices A and B are upper triangular. = .FALSE.: the input matrices A and B are lower triangular. A1 A1 is REAL A2 A2 is REAL A3 A3 is REAL On entry, A1, A2 and A3 are elements of the input 2-by-2 upper (lower) triangular matrix A. B1 B1 is REAL B2 B2 is REAL B3 B3 is REAL On entry, B1, B2 and B3 are elements of the input 2-by-2 upper (lower) triangular matrix B. CSU CSU is REAL SNU SNU is REAL The desired orthogonal matrix U. CSV CSV is REAL SNV SNV is REAL The desired orthogonal matrix V. CSQ CSQ is REAL SNQ SNQ is REAL The desired orthogonal matrix Q. Author: Univ. of Tennessee Univ. of California Berkeley Univ. of Colorado Denver NAG Ltd. Date: September 2012 Definition at line 152 of file slags2.f. Author Generated automatically by Doxygen for LAPACK from the source code. Version 3.4.2 Tue Sep 25 2012 slags2.f(3)
All times are GMT -4. The time now is 09:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy