Extract count of string in all files and display on date wise


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract count of string in all files and display on date wise
Prev   Next
# 1  
Old 11-07-2015
Wrench Extract count of string in all files and display on date wise

Hi All,
hope you all are doing well!
I kindly ask you for shell scripting help, here is the description:

I have huge number of files shown below on date wise, which contains different strings(numbers you can say) including 505001 and 602001.

Code:
exp.interface.1303242.transactions.cor.2015-05-01.002403.tar.gz
transactions.cor.2015-05-01.002403.207051e2a0acff0661020f0506a3463c.tgz
transactions.cor.2015-05-01.002403.chk.812e2b458a6cba3fbbeb0f97731ad986.tgz
exp.interface.1303950.transactions.cor.2015-05-01.002405.tar.gz
transactions.cor.2015-05-01.002405.40e7d04b837c53571944b7c1bc82c4ae.tgz
transactions.cor.2015-05-01.002405.chk.56f70fd05a63775c27345d308b441735.tgz
exp.interface.1304599.transactions.cor.2015-05-01.002407.tar.gz
transactions.cor.2015-05-01.002407.a2948151d638a5eeca4aaed134bd41b9.tgz
transactions.cor.2015-05-01.002407.chk.8ac3628a11d38228402ef314ef49beae.tgz
exp.interface.1305211.transactions.cor.2015-05-01.002409.tar.gz
transactions.cor.2015-05-01.002409.b880f665515e6d18c056ce6483d5ff12.tgz
transactions.cor.2015-05-01.002409.chk.f66026a5bc2810f7a7147e12bf8e1f4e.tgz
exp.interface.1305794.transactions.cor.2015-05-01.002411.tar.gz
transactions.cor.2015-05-01.002411.97595eaf67a0333c40af3f589029f0d7.tgz
transactions.cor.2015-05-01.002411.chk.6fe34a59fff56fa3a3aa03a6fbf1bdea.tgz
exp.interface.1306377.transactions.cor.2015-05-01.002413.tar.gz
transactions.cor.2015-05-01.002413.042a33aa055bdf572529cf23f6fac000.tgz
transactions.cor.2015-05-01.002413.chk.99c1fa81839ab23357d41c09a88d4367.tgz
exp.interface.1306936.transactions.cor.2015-05-01.002415.tar.gz
transactions.cor.2015-05-01.002415.3934512e8bc98df67a34a3f90dc96340.tgz
transactions.cor.2015-05-01.002415.chk.966a39a2493c1b5503547909ee9609fc.tgz
exp.interface.1307493.transactions.cor.2015-05-01.002417.tar.gz
transactions.cor.2015-05-01.002417.4f07864d29eb53dc8eb43f4de4f1e711.tgz
transactions.cor.2015-05-01.002417.chk.6f9bf302c7cf298565f1b8b131afa7f4.tgz
exp.interface.1308085.transactions.cor.2015-05-02.002419.tar.gz
transactions.cor.2015-05-02.002419.eeb2e0d6616c3cf443f77c34c2a27bd5.tgz
transactions.cor.2015-05-02.002419.chk.e3b894ff0eb31786d78a2774d4e7316c.tgz
exp.interface.1308789.transactions.cor.2015-05-02.002421.tar.gz
transactions.cor.2015-05-02.002421.7969fdfb132dabe6c200e2287f685cfe.tgz
transactions.cor.2015-05-02.002421.chk.2a8554aae7caed7c3887f90ca4fd5b8f.tgz
exp.interface.1309434.transactions.cor.2015-05-02.002423.tar.gz
transactions.cor.2015-05-02.002423.678ee6d0f05ce0f727108b0229ef14c2.tgz
transactions.cor.2015-05-02.002423.chk.d8bcb91a5eb5691face8d1f59a26ace5.tgz
exp.interface.1310048.transactions.cor.2015-05-02.002425.tar.gz
transactions.cor.2015-05-02.002425.f49937f7a93af6300ad93ed7bf57423a.tgz
transactions.cor.2015-05-02.002425.chk.d46740d8acc82b221dfa7b27389391c1.tgz
exp.interface.1310626.transactions.cor.2015-05-02.002427.tar.gz
transactions.cor.2015-05-02.002427.6527b1cab4530f8b91d6f6aa9cc37952.tgz
transactions.cor.2015-05-02.002427.chk.304d0babcccb8f293618d59fd8983884.tgz
exp.interface.1311208.transactions.cor.2015-05-02.002429.tar.gz
transactions.cor.2015-05-02.002429.b0ccb8dd11bb106b335915b8fb4985ce.tgz
transactions.cor.2015-05-02.002429.chk.2612b2f02abe419ea774efe75b1495b1.tgz
exp.interface.1311770.transactions.cor.2015-05-02.002431.tar.gz
transactions.cor.2015-05-02.002431.ca996b5cc8f29828ad62b40c390da738.tgz
transactions.cor.2015-05-02.002431.chk.a151e1837ed7ec4d530399bee61be07e.tgz
exp.interface.1312325.transactions.cor.2015-05-02.002433.tar.gz
transactions.cor.2015-05-02.002433.4ba12a47b9f44fe76a760a89d3e6f31e.tgz
transactions.cor.2015-05-02.002433.chk.398df00f78dd0049de322e01fe6746ca.tgz
exp.interface.1312919.transactions.cor.2015-05-03.002435.tar.gz
transactions.cor.2015-05-03.002435.09eeef9eb51974b1527078189c13ef8e.tgz
transactions.cor.2015-05-03.002435.chk.b506f65bf3385486d2170796f762a13e.tgz
exp.interface.1313624.transactions.cor.2015-05-03.002437.tar.gz
exp.interface.1314270.transactions.cor.2015-05-03.002439.tar.gz
transactions.cor.2015-05-03.002437.99f16075cba422ed1d94c74f8d07c2f4.tgz
transactions.cor.2015-05-03.002437.chk.ec7d341056ee26e7415eefa51f4e3288.tgz
transactions.cor.2015-05-03.002439.028a0e7f0721d31caa5441c42aa525de.tgz
transactions.cor.2015-05-03.002439.chk.a15749dec5b30b251e12623799f4dcac.tgz
exp.interface.1314880.transactions.cor.2015-05-03.002441.tar.gz
transactions.cor.2015-05-03.002441.3f550a3c8375bcf30e98cdc61569f608.tgz
transactions.cor.2015-05-03.002441.chk.839c3e5a4412a9dfd08c0066aae8689b.tgz
exp.interface.1315468.transactions.cor.2015-05-03.002443.tar.gz
transactions.cor.2015-05-03.002443.853d1ebc1aca469219eeac4703bbf79b.tgz
transactions.cor.2015-05-03.002443.chk.edad6e9122b540c65df8e8c750825456.tgz
exp.interface.1316054.transactions.cor.2015-05-03.002445.tar.gz
transactions.cor.2015-05-03.002445.5670aa3c41371b4b0fe92c0ab44327c1.tgz
transactions.cor.2015-05-03.002445.chk.bcce7f279b3ae9b4eba37876207ae182.tgz
exp.interface.1316608.transactions.cor.2015-05-03.002447.tar.gz
transactions.cor.2015-05-03.002447.5f9ef688d904286bb9b06af7ede4c0be.tgz
transactions.cor.2015-05-03.002447.chk.0295b8d5a628e23e4f3465a3bfcaa8d1.tgz
exp.interface.1317160.transactions.cor.2015-05-03.002449.tar.gz
transactions.cor.2015-05-03.002449.bcb75d6bd1b2215707f888f624805f55.tgz
transactions.cor.2015-05-03.002449.chk.56f92b5bd0d6380ab191c7e55a3db3e5.tgz
exp.interface.1317760.transactions.cor.2015-05-04.002451.tar.gz
transactions.cor.2015-05-04.002451.6edcbbb58ff42f989c56749ea8f86b80.tgz
transactions.cor.2015-05-04.002451.chk.af02de8f4f0f897ba8b736d766306eeb.tgz
exp.interface.1318464.transactions.cor.2015-05-04.002453.tar.gz
transactions.cor.2015-05-04.002453.a9a94fbf15fadd6a78744501c4a6ba49.tgz
transactions.cor.2015-05-04.002453.chk.f2b0cead4e5456bd86d962674268dfa8.tgz
exp.interface.1319107.transactions.cor.2015-05-04.002455.tar.gz
transactions.cor.2015-05-04.002455.4f43e8a048050da400731d329847f5d7.tgz
transactions.cor.2015-05-04.002455.chk.b9ccf5ad05f922e9d7636cfa7318106e.tgz
exp.interface.1319722.transactions.cor.2015-05-04.002457.tar.gz
transactions.cor.2015-05-04.002457.02aef18177c7ee7019ead0287b958461.tgz
transactions.cor.2015-05-04.002457.chk.16dcdd6e11f7dd0ebe90c08d660e0995.tgz
exp.interface.1320309.transactions.cor.2015-05-04.002459.tar.gz
transactions.cor.2015-05-04.002459.5f51a68060af20116c2930a3e174bf4f.tgz
transactions.cor.2015-05-04.002459.chk.53be659573fb8adbdb08ed2bf360c23b.tgz
exp.interface.1320888.transactions.cor.2015-05-04.002461.tar.gz
transactions.cor.2015-05-04.002461.7963565bd984bccc9cd6b44f95c4fd58.tgz
transactions.cor.2015-05-04.002461.chk.00b7543c2fdf50f8853abb8cd4d616c9.tgz
exp.interface.1321445.transactions.cor.2015-05-04.002463.tar.gz
transactions.cor.2015-05-04.002463.fb298cc433d4b6946d6bb1179c3352ff.tgz
transactions.cor.2015-05-04.002463.chk.3ac335da4046211e1afe88565e5290a1.tgz
exp.interface.1321993.transactions.cor.2015-05-04.002465.tar.gz
transactions.cor.2015-05-04.002465.127e5c7af469c167114b62ef562ce883.tgz
transactions.cor.2015-05-04.002465.chk.1eeb9366c85b2ee6efcc51c3da6c1fa2.tgz
exp.interface.1322596.transactions.cor.2015-05-05.002467.tar.gz
transactions.cor.2015-05-05.002467.41ae0e29651fba731c2501301d655efa.tgz
transactions.cor.2015-05-05.002467.chk.c6321b180a78a94f58d071fbc01e650c.tgz
exp.interface.1323303.transactions.cor.2015-05-05.002469.tar.gz
transactions.cor.2015-05-05.002469.ab4439f269c30c2764d50b74271d45f3.tgz
transactions.cor.2015-05-05.002469.chk.1e8a386198470edcd9cf43ed70a4168e.tgz
exp.interface.1323944.transactions.cor.2015-05-05.002471.tar.gz
transactions.cor.2015-05-05.002471.dd73123909967f907e5692347b01e11b.tgz
transactions.cor.2015-05-05.002471.chk.b3ed1a47580b50e61dfa08f11792f6b6.tgz
exp.interface.1324570.transactions.cor.2015-05-05.002473.tar.gz
transactions.cor.2015-05-05.002473.b81d210295ca5cd25289a68aa112b575.tgz
transactions.cor.2015-05-05.002473.chk.44369c78affd438bc3bfcc3e8a3fd80b.tgz
exp.interface.1325157.transactions.cor.2015-05-05.002475.tar.gz
transactions.cor.2015-05-05.002475.843a216425681d6778894b7812b684f2.tgz
transactions.cor.2015-05-05.002475.chk.153f1c4957fa8f093e59830a66e63473.tgz
exp.interface.1325738.transactions.cor.2015-05-05.002477.tar.gz
transactions.cor.2015-05-05.002477.1137cf1010af69a8b41f6a4ff6189f9a.tgz
transactions.cor.2015-05-05.002477.chk.5bb9e1dfaaf1ff24b73daf76339cd62c.tgz
exp.interface.1326288.transactions.cor.2015-05-05.002479.tar.gz
transactions.cor.2015-05-05.002479.8b088f749460b93a183799d417b0092c.tgz
transactions.cor.2015-05-05.002479.chk.a88dda5fcaa4fa33a59cfae969dc5e1c.tgz
exp.interface.1326834.transactions.cor.2015-05-05.002481.tar.gz
transactions.cor.2015-05-05.002481.0ee0bfaea345c3167ed7b30ff34a98d9.tgz
transactions.cor.2015-05-05.002481.chk.b9ae8b670e091cd75b3a6ce9c4b3c036.tgz
exp.interface.1327436.transactions.cor.2015-05-06.002483.tar.gz
transactions.cor.2015-05-06.002483.af6549f393d27dc78a7544f42d6dd570.tgz
transactions.cor.2015-05-06.002483.chk.26b3627a58b70d82bf666d9cbc4e7d98.tgz
exp.interface.1328144.transactions.cor.2015-05-06.002485.tar.gz
exp.interface.1328781.transactions.cor.2015-05-06.002487.tar.gz
transactions.cor.2015-05-06.002485.2bd23fb03fe6b48b4110a35b40da7dd0.tgz
transactions.cor.2015-05-06.002485.chk.222dcabc5ccfa6c0fe61b13be1ac3b06.tgz
transactions.cor.2015-05-06.002487.cec42a84aaf31048713bc83ffa78a2c3.tgz
transactions.cor.2015-05-06.002487.chk.417aea0ad4f733f005462785cf8b8aa3.tgz
exp.interface.1329384.transactions.cor.2015-05-06.002489.tar.gz
transactions.cor.2015-05-06.002489.9c4b19e84be4d1d1482cfbac473d7764.tgz
transactions.cor.2015-05-06.002489.chk.0f914c2e77efee534746acb03e576618.tgz
exp.interface.1329972.transactions.cor.2015-05-06.002491.tar.gz
transactions.cor.2015-05-06.002491.8b9db656a66629b47219f979418c6541.tgz
transactions.cor.2015-05-06.002491.chk.d9f756faef0c67bac6099d82f1a7b98c.tgz
exp.interface.1330551.transactions.cor.2015-05-06.002493.tar.gz
transactions.cor.2015-05-06.002493.13ef91ca2fada5ecc972011bfff6fbed.tgz
transactions.cor.2015-05-06.002493.chk.f8a0f747daafda74f5559fed9455d787.tgz
exp.interface.1331106.transactions.cor.2015-05-06.002495.tar.gz
transactions.cor.2015-05-06.002495.665139f339f29ff7fdcaaf2a5c56bcc9.tgz
transactions.cor.2015-05-06.002495.chk.033689678f3bffdd1c78be2064cb3681.tgz
exp.interface.1331666.transactions.cor.2015-05-06.002497.tar.gz
transactions.cor.2015-05-06.002497.a925363d76a3d97ac364544ae4118679.tgz
transactions.cor.2015-05-06.002497.chk.a2f855be7d6ad0cbde459154c7ad6f84.tgz
exp.interface.1332271.transactions.cor.2015-05-07.002499.tar.gz
transactions.cor.2015-05-07.002499.adafe2b73a0c5c653046672131dc021f.tgz
transactions.cor.2015-05-07.002499.chk.1109302d88f1da586925698df2d10a7d.tgz
exp.interface.1332978.transactions.cor.2015-05-07.002501.tar.gz
transactions.cor.2015-05-07.002501.7b557e32b07ae9a5105c365c8517f147.tgz
transactions.cor.2015-05-07.002501.chk.b62afb04835adac1e247007508a7ad37.tgz
exp.interface.1333623.transactions.cor.2015-05-07.002503.tar.gz
transactions.cor.2015-05-07.002503.7b1a24ce9847ede90c412d1c121b4144.tgz
transactions.cor.2015-05-07.002503.chk.854720b16cea034d14d0d52295631191.tgz
exp.interface.1334242.transactions.cor.2015-05-07.002505.tar.gz
transactions.cor.2015-05-07.002505.254ded0792208442326837f4219a3c51.tgz
transactions.cor.2015-05-07.002505.chk.d4c91b98bf4c7bce0a13cb5ca3637cd5.tgz
exp.interface.1334837.transactions.cor.2015-05-07.002507.tar.gz
transactions.cor.2015-05-07.002507.fc0a2fd466d34298334b9208453d371e.tgz
transactions.cor.2015-05-07.002507.chk.7e7df789b4fb9640df6cf29e55335059.tgz
exp.interface.1335410.transactions.cor.2015-05-07.002509.tar.gz
transactions.cor.2015-05-07.002509.23137badbc153e06945f0b1733c4ea20.tgz
transactions.cor.2015-05-07.002509.chk.18607fc2043a8ee2d86e9888157df0b5.tgz
exp.interface.1335960.transactions.cor.2015-05-07.002511.tar.gz
transactions.cor.2015-05-07.002511.945eb9dffe79e6be1852e3ded2e87a9a.tgz
.
.
.
...2015-10-31...
...2015-10-31...
...2015-10-31...
...2015-10-31...
...2015-10-31...

I can straight a way ignore the files starting with exp and transactions files which contains .chk.
In rest of all other files starting from ...2015-05-01... to ...2015-10-31..., i need the count of string (or number) 505001 and 602001 and display on date wise. here i am using the shell script, but some where i am missing the logic.
Code:
#!/bin/bash
count_505001=0
count_602001=0
tempdate=${file:-"2015-05-01"}
for file in transactions.cor.2015-0[56789]; do 
	if [[ $tempdate = ${file:18:10} ]]; then
		tempdate=${file:18:10}
		count_505001=count+`zless $file | egrep "505001" | wc -l`;
		count_602001=count+`zless $file | egrep "602001" | wc -l`;
	else
		echo "Count for 505001 :"$count_505001" on date: "$tempdate
		echo "Count for 602001 :"$count_602001" on date: "$tempdate
		tempdate=${file:18:10}
		count_505001=`zless $file | egrep "505001" | wc -l`;
		count_602001=`zless $file | egrep "602001" | wc -l`;
	fi
done;

here is the example output:
Code:
Count for 505001 :236 on date: 2015-05-01
Count for 602001 :371 on date: 2015-05-01

Count for 505001 :9 on date: 2015-05-02
Count for 602001 :42 on date: 2015-05-02

Count for 505001 :53 on date: 2015-05-03
Count for 602001 :63 on date: 2015-05-03
.
.
.
Count for 505001 :423 on date: 2015-10-31
Count for 602001 :745 on date: 2015-10-31

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search string in multiple files and display column wise

I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number File1 Name = Joe Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies

2. Shell Programming and Scripting

Count the number of string occurrences to display 0 entries in output

Hello Friends, Can somebody assist an issue I am having? I have a separate file with a list of account ids XXX200B02Y01 XXX200B03Y01 XXX200B05Y01 XXX200B07Y01 XXX200B08Y01 I call the file, and run an egrep against a directory and logfiles AccountID=$(cat... (2 Replies)
Discussion started by: liketheshell
2 Replies

3. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

4. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

5. Shell Programming and Scripting

How to read files by Server Creation date wise?

Hi All, I would have many files in the server with xyz*.dat -- Static file name Physical files: xyz1.dat - 01PM xyz2.dat - 02PM xyz3.dat - 03PM In present version we are using for f in $file_name do fname=`ls $f | grep -v ^'\|'$ | sed s/' '/'\\ '/g` .... sqlldr... (4 Replies)
Discussion started by: Dharv
4 Replies

6. Shell Programming and Scripting

Extract string from multiple file based on line count number

Hi, I search all forum, but I can not find solutions of my problem :( I have multiple files (5000 files), inside there is this data : FILE 1: 1195.921 -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05 -2021.287 1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05 ... (15 Replies)
Discussion started by: guns
15 Replies

7. Solaris

delete files by date wise

Hi guys, I want to delete files from june 13 to june 30, using rm command can any one tell me the sintax to remove. I ahve hunderd of core files in my /var dir. so i want to clear last month core files. Thanks in Advance.:)) (2 Replies)
Discussion started by: kurva
2 Replies

8. Shell Programming and Scripting

Need to zip the files date wise --urgent Help needed

Hi all, this is my first post, i need to write a script to zip the files with datewise below are the log files. -rw------- 1 root sso 85316156 May 24 22:11 core_test_smservaz_104_104_1243217459_8896 -rw------- 1 root sso 90413304 May 25 22:12 core_test_smservaz_104_104_1243303895_20912... (4 Replies)
Discussion started by: lcschandu
4 Replies

9. Shell Programming and Scripting

To extract data of a perticular interval (date-time wise)

I want a shell script which extract data from a log file which contains date and time-wise data and i need the data for a perticular interval of time...what can i do??? (3 Replies)
Discussion started by: abhishek27
3 Replies

10. Shell Programming and Scripting

Display the count of files

I am new to shell programming. Can anyone help me out with anyone of these? Display a count of the number of regular files, the number of symbolic links, the number of sub-directories, the number of block-special files, and the number of character-special files in the directory. I don't... (4 Replies)
Discussion started by: wayne1411
4 Replies
Login or Register to Ask a Question