Get counts for multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get counts for multiple files
# 1  
Old 08-27-2010
Get counts for multiple files

How do get the counts by excluding header and tailer.
Code:
 wc -l customer_data*.0826
      31 customer_data_1.0826
      57 customer_data_2.0826
     456 customer_data_3.0826
     668 customer_data_4.0826
     789 customer_data_5.0826
    2344 customer_data_6.0826
   13457 customer_data_7.0826
   18112 customer_data_8.0826
   24876 customer_data_9.0826

# 2  
Old 08-27-2010
Try this:
Code:
fileCount=0
wc -l customer_data*.0826 | while read lineCount FileName
do
      lineCountNHT=`expr ${lineCount} - 2`
      echo "FileName: [${FileName}] - LineCount: [${lineCountNHT}]".
done

I hope it helps!

Regards.

---------- Post updated at 13:22 ---------- Previous update was at 13:04 ----------

Change:
Code:
# From: wc -l customer_data*.0826 - To:
wc -l customer_data*.0826 | egrep -v 'total$'

# 3  
Old 08-27-2010
Code:
 
for file in `ls -1 customer_data*.0826 `; do echo "$(expr `wc -l<$file` - 2 ) "$file"" ; done

# 4  
Old 08-27-2010
Another way:
Code:
for mFile in `ls -1 customer_data*.0826`; do perl -lne '{$l++}; END {print "File: [$ARGV] [".($l-2)."]"}' $mFile; done

# 5  
Old 08-28-2010
Thank You felipe and ygemici. That works. Since the file names remain constant ( the date alone changes for every three days) how can we modify the code to run these scripts for every three days?
# 6  
Old 08-28-2010
If I understood your question correctly, I would schedule a crontab job and add a variable to the date in the file name.

Something like:
Code:
# customer_data*.MMDD
currDateMMDD=`date '+%m%d'`
for mFile in `ls -1 customer_data*.${currDateMMDD}`; do perl -lne '{$l++}; END {print "File: [$ARGV] [".($l-2)."]"}' $mFile; done

About the crontab, check these links:
- cron - Wikipedia, the free encyclopedia
- Introduction to UNIX cron and at Utilities
# 7  
Old 08-28-2010
vinturin thanks for your updates. My way of testing also same. But now i struked up with comparing the file count. I want to disaply the count only if all customer files exits in server with data say 10 files.
The variable files_total_cnt should have value 10 then get the counts from mall 10 files.

Code:
 #!/bin/ksh
path=/folder1/folder2/folder3/folder4/folder5/
cd $path
actualdate=`date +"%Y%m%d" -d "2 day ago"`
echo $actualdate
files_total_cnt=$(ls -l mcvp_fmc_usa_*.$actualdate) | some thing trying to get actual file cnt.....
if files_total_cnt > 10
then
for file in `ls -1 customer_data*.actualdate`; do echo "$(expr `wc -l<$file` - 2 ) "$file"" ; done
else
echo "some files are missing"
fi

note am getting count for 2 days old file so i coded in that way.

---------- Post updated at 02:53 PM ---------- Previous update was at 02:24 PM ----------

I got the solution. Thanks all.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

2. Shell Programming and Scripting

Run one script on multiple files and print out multiple files.

How can I Run one script on multiple files and print out multiple files. FOR EXAMPLE i want to run script.pl on 100 files named 1.txt ....100.txt under same directory and print out corresponding file 1.gff ....100.gff.THANKS (4 Replies)
Discussion started by: grace_shen
4 Replies

3. UNIX for Dummies Questions & Answers

Run one script on multiple files and print out multiple files.

How can I run the following command on multiple files and print out the corresponding multiple files. perl script.pl genome.gff 1.txt > 1.gff However, there are multiples files of 1.txt, from 1----100.txt Thank you so much. No duplicate posting! Continue here. (0 Replies)
Discussion started by: grace_shen
0 Replies

4. Shell Programming and Scripting

Column extraction from multiple files to multiple files

I have roughly ~30 .txt files in a directory which all have unique names. These files all contain text arranged in columns separated by whitespace (example file: [#YY MM DD hh mm WDIR WSPD GST WVHT DPD APD MWD PRES ATMP WTMP DEWP VIS TIDE #yr mo dy hr mn degT m/s m/s m sec ... (5 Replies)
Discussion started by: aozgaa
5 Replies

5. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

6. Shell Programming and Scripting

Need to combine counts from diff files

Ok i have a script that goes into the last 3 files, grabs a value from that file which is a date, counts the number of results of that date. So this is the output: FILED001.20110407.175709.086912 49995 04-07T12 4 04-07T07 1 04-07T17 FILED001.20110407.180111.086913 50000 04-07T12 ... (5 Replies)
Discussion started by: cinderella
5 Replies

7. Shell Programming and Scripting

Sending e-mail of record counts in 3 or more files

I am trying to load data into 3 tables simultaneously (which is working fine). Then when loaded, it should count the total number of records in all the 3 input files and send an e-mail to the user. The script is working fine, as far as loading all the 3 input files into the database tables, but... (3 Replies)
Discussion started by: msrahman
3 Replies

8. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

9. Shell Programming and Scripting

how to compare counts in two separate files

Hi all, what will be the code to compare count present in two seperate files for e.g file (a) contains counts 100 and file (b) contains records 90 since both these files have differnt count so it will display count didnt match and in case of success it display (5 Replies)
Discussion started by: jojo123
5 Replies

10. Shell Programming and Scripting

Comparing Counts Within Separate Files

Hey all, So I have this challenge where I am attempting to compare record counts from within several different log files. I want input and output counts for each file, and I want to compare that with the result of the input/output comparison from a separate--but related file. Example: ... (2 Replies)
Discussion started by: gator76
2 Replies
Login or Register to Ask a Question