Sponsored Content
Full Discussion: Record count of a csv file
Top Forums Shell Programming and Scripting Record count of a csv file Post 302215965 by ajaykk on Thursday 17th of July 2008 03:43:00 PM
Old 07-17-2008
Quote:
Originally Posted by joeyg
1) cat file | grep "^[0-9][0-9]" | wc -l
Or similar, if the normal file has numbers in the first couple character positions as user id or other identifier

2) cat file | cut -c1 | grep '"'
Assuming that the first field begins with a "

3) cat sample | tr "\n" " " | tr '"' "\n" | wc -l
Will produce a number. If normally three fields with " for each valid record, then take the resulting number from the command and divide by (twice the normal # fields).
The command above showed me 12 for my sample file, and I knew my sample has three fields with ". So 12/(3*2) = 2 records


p.s.
Be cautious of the "bump" to move a question. I always scan questions for those with zero responses - to work on first. Then, time permitting, go back to others that I previously skipped over. Therefore, your "bump" can have the reverse effect.

Thanks Joey..
cat sample | tr "\n" " " | tr '"' "\n" | wc -l..seems to be working..
when I tried..found an issue..for one of the record the data has multiple double quotes ..("xyz","abc","toys ""R"" us", "def", "hij","....)..
coz of this I'm not getting a round value count..how do I handle this?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to delete a record from a csv file

Hi Guys I have downloaded a table from oracle database in .csv format. it has many fields as Title, First Name, Last Name etc. I have to download distinct titles from database and now i have to check all those titles from data of First Name one by one. and then i have to delete matched record.... (1 Reply)
Discussion started by: Rajeev Agrawal
1 Replies

2. UNIX for Dummies Questions & Answers

How to count the record count in an EBCDIC file.

How do I get the record count in an EBCDIC file on a Linux Box. :confused: (1 Reply)
Discussion started by: oracle8
1 Replies

3. UNIX for Dummies Questions & Answers

how to get a file name & record count of csv file

HI , I am new to shell scripting , I have a requirement that I send a file for data quality ( original.csv) & i will be getting 4 files daily into a particular directory in return with cleansed data . the files may be clean.csv, unclean.csv , ... (2 Replies)
Discussion started by: sirik
2 Replies

4. Shell Programming and Scripting

replaying a record count with another record count

i use unix command to take the record count for a file1 awk 'END{print NR}' filename i already have a file2 which conatin the count like ... .. rec_cnt=100 .. .. I want to replace the record in the file2 using the record i take from file1. suggest me some simple ways of doing it... (2 Replies)
Discussion started by: er_zeeshan05
2 Replies

5. Shell Programming and Scripting

Split long record into csv file

Hi I receive a mainframe file which has very long records (1100 chars) with no field delimiters. I need to parse each record and output a comma delimited (csv) file. The record layout is fixed. If there weren't so many fields and records I would read the file into Excel, as a "fixed width"... (10 Replies)
Discussion started by: wvdeijk
10 Replies

6. Shell Programming and Scripting

csv file - adding total to a trailer record

Hi, I have a script which creates and modifies a csv file. I have managed to do everything I need to do apart from 1 thing. I need to append a trailer record to the file. I need this line to hold the total of an entire column of the csv file (skipping the 1st line which is a header). Can... (2 Replies)
Discussion started by: mcclunyboy
2 Replies

7. Shell Programming and Scripting

Print record count of a file using shell script

HI, I need to print the record count of a file using shell script. If the record count of a file excluding header and trailer record if greater than zero then print 'Record count of a file is xxxx records'. If the record count is zero print 'zero records' Thanks Mahendra (1 Reply)
Discussion started by: mmeda
1 Replies

8. Shell Programming and Scripting

Output first unique record in csv file

Hi, I have to output a new csv file from an input csv file with first unique value in the first column. input csv file color product id status green 102 pass yellow 201 hold yellow 202 keep green 101 ok green 103 hold yellow 203 ... (5 Replies)
Discussion started by: Chris LAU
5 Replies

9. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

10. Shell Programming and Scripting

Total record count of all the file present in a directory

Hi All , We need one help on the below requirement.We have multiple pipe delimited .txt file(around 100 .txt files) present on one directory.We need the total record count of all the files present in that directory without header.File format as below : ... (8 Replies)
Discussion started by: STCET22
8 Replies
cut(1)							      General Commands Manual							    cut(1)

NAME
cut - Displays specified parts from each line of a file SYNOPSIS
cut -b list [-n] [file...] cut -c list [file...] cut -f list [-d delim] [-s] [file...] STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: cut: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Cuts based on a list of bytes. Each selected byte is output, unless you also specify the -n option. For example, if you specify -b 1-72, the cut command writes out the first 72 bytes in each line of the file. Cuts based on a list of characters. It is not an error if you specify a character not in the input. Uses the specified character as the field delimiter (separator) when you specify the -f option. You must quote characters with special meaning to the shell, such as the space character. Any character can be used as delim. The default field delimiter is a tab character. Specifies a list of fields assumed to be separated in the file by a field delimiter character, speci- fied by the -d option or the tab character by default. For example, if you specify -f 1,7, the cut command writes out only the first and seventh fields of each line. If a line contains no field delimiters, the cut command passes them through intact (useful for table subhead- ings), unless you specify the -s option. Does not split characters. When specified with the -b option, each element in list of the form low-high (hyphen-separated numbers) is modified as follows: If the byte selected by low is not the first byte of a character, low is decre- mented to select the first byte of the character originally selected by low. If the byte selected by high is not the last byte of a char- acter, high is decremented to select the last byte of the character prior to the character originally selected by high, or zero (0) if there is no prior character. If the resulting range element has high equal to zero (0) or low greater than high, the list element is dropped from list for that input line without causing an error. Each element in list of the form low- is treated as previously described with high set to the number of bytes in the current line, not including the terminating newline character. Each element in list of the form -high is treated as previously described with low set to 1. Each element in list of the form number (a single number) is treated as previously described with low set to number and high set to number. Suppresses lines that do not contain delimiter characters (use only with the -f option). Unless you include this option, lines with no delimiters are passed through. OPERANDS
The path name of the file to be examined. If you do not specify a file or you specify a hyphen (-), the cut command reads standard input. DESCRIPTION
The cut command locates the specified fields in each line of the specified file and writes the characters in those fields to standard out- put. You must specify the -b option (to select bytes), the -c option (to select characters) or the -f option (to select fields). The list argu- ment (see the -b, -c, and -f options) must be a space-separated or comma-separated list of positive numbers and ranges. Ranges can be in three forms: Two positive numbers separated by a hyphen (-), as in the form low-high, which represents all fields from the first number to the second number. A positive number preceded by a hyphen (-), as in the form -high, which represents all fields from field number 1 to that number. A positive number followed by a hyphen (-), as in the form low-, which represents that number to the last field, inclusive. The elements in list can be repeated, can overlap, and can be specified in any order. Some sample list specifications are as follows: First, fourth, and seventh bytes or fields. First through third and eighth bytes or fields. First through fifth and tenth bytes or fields. Third through last bytes or fields. The fields specified by list can be a fixed number of byte positions, or the length can vary from line to line and be marked with a field delimiter character, such as a tab character. [Tru64 UNIX] You can also use the grep command to make horizontal cuts through a file and the paste command to put the files back together. To change the order of columns in a file, use the cut and paste commands. EXIT STATUS
The following exit values are returned: Successful completion. An error occurred. EXAMPLES
To display several fields of each line of a file, enter: cut -f 1,5 -d : /etc/passwd This displays the login name and full user name fields of the system password file. These are the first and fifth fields (-f 1,5) sepa- rated by colons (-d :). So, if the /etc/passwd file looks like this: su:UHuj9Pgdvz0J":0:0:User with special privileges:/: daemon:*:1:1::/etc: bin:*:2:2::/usr/bin: sys:*:3:3::/usr/src: adm:*:4:4:System Admin- istrator:/usr/adm: pierre:*:200:200:Pierre Harper:/u/pierre: joan:*:202:200:Joan Brown:/u/joan: Then, cut -f 1,5 -d : /etc/passwd produces this output: su:User with special privileges daemon: bin: sys: adm:System Administrator pierre:Pierre Harper joan:Joan Brown ENVIRONMENT VARIABLES
The following environment variables affect the execution of cut: Provides a default value for the internationalization variables that are unset or null. If LANG is unset or null, the corresponding value from the default locale is used. If any of the internationalization vari- ables contain an invalid setting, the utility behaves as if none of the variables had been defined. If set to a non-empty string value, overrides the values of all the other internationalization variables. Determines the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multibyte characters in arguments and input files). Determines the locale for the format and contents of diagnostic messages written to standard error. Determines the location of message catalogues for the processing of LC_MESSAGES. SEE ALSO
Commands: grep(1), fold(1), join(1), paste(1) Standards: standards(5) cut(1)
All times are GMT -4. The time now is 04:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy