Shell script for field wise record count for different Files .csv files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script for field wise record count for different Files .csv files
# 1  
Old 01-21-2016
Shell script for field wise record count for different Files .csv files

Hi,

Very good wishes to all!

Please help to provide the shell script for generating the record counts in filed wise from the .csv file

My question:

Source file:

Code:
Field1 Field2 Field3
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn

My target file should generate the .csv file with the record count on field wise as below:


Target File:

Code:
Field1 Field2 Field3
4 3 2

Field1 has 4 values and field2 has 3 values and field3 has two values so the target file should be as below.

My source file can changes with more columns or less, so the target file should also have the same number of columns with the record count. Please help to resolve this..

Regards,
Kiran.

Last edited by Don Cragun; 01-21-2016 at 05:39 AM.. Reason: Add CODE tags.
# 2  
Old 01-21-2016
Hello Kirands,

Please use code tags for code/Input/commands which you are using into you posts as per forum rules. Following may help you in same.
Code:
awk 'NR==1{print;next} {for(i=1;i<=NF;i++){array[i]++}} END{for(j in array){A=A?A "\t" array[j]:array[j]};print A}'   Input_file

Output will be as follows.
Code:
Field1 Field2 Field3
4       3       2

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 3  
Old 01-21-2016
Hi Ravinder,

Thank you so much for your response.
The data is looking good when i run the code given by you.. can you please help to put that data in the .csv file with the comma separator.

Regards,
Kiran.
# 4  
Old 01-21-2016
Hello Kirands,

Sorry forgot to mention that previously, following may help you in same.
Code:
awk 'NR==1{OFS=",";for(i=1;i<=NF;i++){Q=Q?Q OFS $i:$i}print Q;next} {for(i=1;i<=NF;i++){array[i]++}} END{for(j in array){A=A?A OFS array[j]:array[j]};print A}' OFS=,  Input_file > Output.csv

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 01-21-2016
Hi Ravinder,

Thank you for your prompt response.

I have tested the code given by you, but its still have some issues in giving the correct record count.

Please find the attached source file that has the data and the two headers., the record count should be from the 3rd row.
Please help to resolve this issue.

Thanks for your support.

Regards,
Kiran.
# 6  
Old 01-21-2016
Hello Kirands,

I am sorry, I couldn't open the .rar file. According to your post you need to leave first two lines then let's say we have following Input_file:
Code:
Field1 Field2 Field3
XYZ ABC DEF
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn

Then a little change in my previous post code as follows.
Code:
awk 'NR==1 || NR==2{OFS=",";for(i=1;i<=NF;i++){Q=Q?Q OFS $i:$i}print Q;Q="";next} {for(i=1;i<=NF;i++){array[i]++}} END{for(j in array){A=A?A OFS array[j]:array[j]};print A}' OFS=,  Input_file > output.csv

Let me know if you have any queries here.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 7  
Old 01-21-2016
Hi Ravindra,

Thank you for your response.

I have executed the latest code send by you :
Code:
awk 'NR==1 || NR==2{OFS=",";for(i=1;i<=NF;i++){Q=Q?Q OFS $i:$i}print Q;Q="";next} {for(i=1;i<=NF;i++){array[i]++}} END{for(j in array){A=A?A OFS array[j]:array[j]};print A}' OFS=,  RecountCountTest.csv> example_4.csv

The output data is giving correct for first 4 columns and the last 5 columns are not giving the correct record count, below are the source file and the target file(after i executed the above code) for your reference:

Source file:
First two are the headers and should not be in the count working good:

Image


Target file after i executed the code provided by you:

Image

for the columns NICKNAME count should be 0

Please help to resolve this issue, thanks for all your support.

Regards,
Kiran.

---------- Post updated at 07:52 AM ---------- Previous update was at 07:49 AM ----------

Hi Ravinder,

Here is the source data as per my previous reply if you did not able to see the data:

Source data:

STATUS USERID USERNAME FIRSTNAME NICKNAME MI LASTNAME SUFFIX TITLE
STATUS USERID Username FirstName Nickname MiddleName LastName Suffix Title
ACTIVE 80001001 itria1 Carlos Guillermo Itria
ACTIVE 80001002 lafournere1 Jean-Paul
ACTIVE 80001003 Wade Peter
ACTIVE 80001004 Olivier

Target Data: (after i executed the code provided by youSmilie

STATUS USERID USERNAME FIRSTNAME NICKNAME MI LASTNAME SUFFIX TITLE
STATUS USERID Username FirstName Nickname MiddleName LastName Suffix Title
4 4 2 4 4 4


Regards,
Kiran.

Last edited by vbe; 01-21-2016 at 09:12 AM.. Reason: code tags!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding total distinct count from multiple csv files through UNIX script

Hi All , I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct count on all files. We can't merge all the files here as file size are huge in millions.I have tried in below way for each... (9 Replies)
Discussion started by: STCET22
9 Replies

2. Shell Programming and Scripting

Extract count of string in all files and display on date wise

Hi All, hope you all are doing well! I kindly ask you for shell scripting help, here is the description: I have huge number of files shown below on date wise, which contains different strings(numbers you can say) including 505001 and 602001. ... (14 Replies)
Discussion started by: VasuKukkapalli
14 Replies

3. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

4. Shell Programming and Scripting

How to create or convert to pdf files from csv files using shell script?

Hi, Can anyone help me how to convert a .csv file to a .pdf file using shell script Thanks (2 Replies)
Discussion started by: ssk250
2 Replies

5. Shell Programming and Scripting

Record count checking for multiple files through for-loop

Hi Friends, I wrote one shell script to check the record count in two files and that will send us the notification activity if found zero record count. What i did is I created for loop and checking the count for both of the files but what is happening is for first file has data then it's... (13 Replies)
Discussion started by: victory
13 Replies

6. Shell Programming and Scripting

Script to compare count of two csv files

Hi Guys, I need to write a script to compare the count of two csv files each having 5 columns. Everyday a csv file is recived. Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Discussion started by: Vivekit82
3 Replies

7. Shell Programming and Scripting

count of record in files

Hi all, I have written a scripts which count number of lines in all the files in a directory and write in a text file. I don't know how to format it while writing. OS suns solaris 10 my scripts is as below for i in /ersdg3/ERS/ERS_INPUT_LOGS/RIO/LOGS/RIO_02-Aug-2012/ *.LOG do echo... (11 Replies)
Discussion started by: guddu_12
11 Replies

8. Shell Programming and Scripting

Combine Multiple text or csv files column-wise

Hi All I am trying to combine columns from multiple text files into a single file using paste command but the record length being unequal in the different files the data is running over to the closest empty cell on the left. Please see below. What can i do to resolve this ? File 1 File... (15 Replies)
Discussion started by: venky_ibm
15 Replies

9. Shell Programming and Scripting

Comparison of record count of two files

Hi, I have one text file and zip file in UNIX directory.The Zip file contains another text file. For Ex: Text File Name = Req_file.txt Zip file Name= Response_file.txt_04072009_121548.gz Zip file contains one text file that name is Response_file.txt I want to compare the record... (1 Reply)
Discussion started by: praka
1 Replies
Login or Register to Ask a Question