Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Counting lines in multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Counting lines in multiple files
# 1  
Old 11-10-2009
Counting lines in multiple files
Hi,
I have couple of .txt files (say 50 files) in a folder.

For each file:
I need to get the number of lines in each file and then that count -1 (I wanted to exclude the header.

Then sum the counts of all files and output the total sum.

Is there an efficient way to do this using shell programming or awk.

Please let me know.

LA
# 2  
Old 11-10-2009
Use gawk, nawk or /usr/xpg4/bin/awk on Solaris:

Code:
awk 'END{print NR-(ARGC-1)}' *

# 3  
Old 11-10-2009
Sorry I made a mistake in the above post.

Your reply was good. but it was not actually I needed.

For each file can we modify the above awk command to get the number of lines - header for each file and out put this in to another text file.

ie. the output file will be number of lines - 1 for each file

Please let me know.

LA
# 4  
Old 11-10-2009
Something like:
Code:
awk '{a[FILENAME]++}END{for(i in a)print i,(a[i]-1)}' * > newfile

This User Gave Thanks to danmero For This Post:
Elly
# 5  
Old 11-10-2009
I modified Danmero's code slightly to also include totals:
Code:
awk '{a[FILENAME]++}END{for(i in a){r=a[i]-1;t+=r;print i,r}print "total "t}' *.txt

Alternative 1 using pipe:
Code:
grep -c '' *.txt|awk -F: '{$2-=1;t+=$2;print} END{print "Total "t}'

Alternative 2 using pipe:
Code:
 wc -l *.txt|awk '$2=="total"{$1+=1-n}{$1-=1; n++}1'

# 6  
Old 11-10-2009
And a variation of Scrutinizer's last one-liner:

Code:
wc -l * | perl -lne '@x=split; print /total/ ? $x[0]-$.+1 : $x[0]-1," ",$x[1]'

tyler_durden
# 7  
Old 11-10-2009
And another one:

Code:
wc -l * | awk '(/total/&&$1-=NR-1)||--$1'

It doesn't handle 0 record files correctly Smilie

Or:
Code:
wc -l *|perl -pe's/(\d+)/$x=$1;m|total|?$x-($.-1):$1-1/e'

Last edited by radoulov; 11-10-2009 at 06:07 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

2. Shell Programming and Scripting

[Solved] Counting The Number of Lines Between Values with Multiple Variables

Hey everyone, I have a bunch of lines with values in field 4 that I am interested in. If these values are between 1 and 3 I want it to count all these values to all be counted together and then have the computer print out LOW and the number of lines with those values in between 1 and 3,... (2 Replies)
Discussion started by: VagabondGold
2 Replies

3. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

4. Shell Programming and Scripting

Counting occurrences of all words in multiple files

Hey Unix gurus, I would like to count the number occurrences of all the words (regardless of case) across multiple files, preferably outputting them in descending order of occurrence. This is well beyond my paltry shell scripting ability. Researching, I can find many scripts/commands that... (4 Replies)
Discussion started by: twjolson
4 Replies

5. Shell Programming and Scripting

multiple files: counting

In a directory, I have 5000 multiple files that contains around 4000 rows with 10 columns in each file containing a unique string 'AT' located at 4th column. OM 3328 O BT 268 5.800 7.500 4.700 0.000 1.400 OM 3329 O BT 723 8.500 8.900... (7 Replies)
Discussion started by: asanjuan
7 Replies

6. Shell Programming and Scripting

pattern search for multiple log files and counting

I have 10 appservers and each appserver has 4 jvms . Each of these logs is archived and stored on a nfs directory . For example the files are /logs/200907/ap1-jvm1.server.log.20090715.gz /logs/200907/ap2-jvm2.server.log.20090714.gz /logs/200908/ap1-jvm1.server.log.20090812.gz I want to... (3 Replies)
Discussion started by: gubbu
3 Replies

7. Shell Programming and Scripting

counting the lines of diff files in loop

i have two file. i want to count the lines of each file one by one in loop and compare it. can any one pls help me on this? (1 Reply)
Discussion started by: Aditya.Gurgaon
1 Replies

8. Shell Programming and Scripting

replace multiple lines in multiple files

i have to search a string and replace with multiple lines. example Input echo 'sample text' echo 'college days' output echo 'sample text' echo 'information on students' echo 'emp number' echo 'holidays' i have to search a word college and replace the multiple lines i have... (1 Reply)
Discussion started by: unihp1
1 Replies

9. Shell Programming and Scripting

Merging files with AWK filtering and counting lines

Hi there, I have a couple of files I need to merge. I can do a simple merge by concatenating them into one larger file. But then I need to filter the file to get a desired result. The output looks like this: TRNH 0000000010941 ORDH OADR OADR ORDL ENDT 1116399 000000003... (2 Replies)
Discussion started by: Meert
2 Replies

10. UNIX for Dummies Questions & Answers

Counting lines and files

Hi experts, First of all thanks for all your help. How can i count the lines within a text file and send this number to another text file? And by the way how can i count the number of files inside a tape ("/dev/rtp") that as one pattern (Ex. "/CTA/") and send this number to a text file? I... (6 Replies)
Discussion started by: jorge.ferreira
6 Replies
Login or Register to Ask a Question