11-07-2015
Extract count of string in all files and display on date wise
Hi All,
hope you all are doing well!
I kindly ask you for shell scripting help, here is the description:
I have huge number of files shown below on date wise, which contains different strings(numbers you can say) including 505001 and 602001.
Code :
exp.interface.1303242.transactions.cor.2015-05-01.002403.tar.gz
transactions.cor.2015-05-01.002403.207051e2a0acff0661020f0506a3463c.tgz
transactions.cor.2015-05-01.002403.chk.812e2b458a6cba3fbbeb0f97731ad986.tgz
exp.interface.1303950.transactions.cor.2015-05-01.002405.tar.gz
transactions.cor.2015-05-01.002405.40e7d04b837c53571944b7c1bc82c4ae.tgz
transactions.cor.2015-05-01.002405.chk.56f70fd05a63775c27345d308b441735.tgz
exp.interface.1304599.transactions.cor.2015-05-01.002407.tar.gz
transactions.cor.2015-05-01.002407.a2948151d638a5eeca4aaed134bd41b9.tgz
transactions.cor.2015-05-01.002407.chk.8ac3628a11d38228402ef314ef49beae.tgz
exp.interface.1305211.transactions.cor.2015-05-01.002409.tar.gz
transactions.cor.2015-05-01.002409.b880f665515e6d18c056ce6483d5ff12.tgz
transactions.cor.2015-05-01.002409.chk.f66026a5bc2810f7a7147e12bf8e1f4e.tgz
exp.interface.1305794.transactions.cor.2015-05-01.002411.tar.gz
transactions.cor.2015-05-01.002411.97595eaf67a0333c40af3f589029f0d7.tgz
transactions.cor.2015-05-01.002411.chk.6fe34a59fff56fa3a3aa03a6fbf1bdea.tgz
exp.interface.1306377.transactions.cor.2015-05-01.002413.tar.gz
transactions.cor.2015-05-01.002413.042a33aa055bdf572529cf23f6fac000.tgz
transactions.cor.2015-05-01.002413.chk.99c1fa81839ab23357d41c09a88d4367.tgz
exp.interface.1306936.transactions.cor.2015-05-01.002415.tar.gz
transactions.cor.2015-05-01.002415.3934512e8bc98df67a34a3f90dc96340.tgz
transactions.cor.2015-05-01.002415.chk.966a39a2493c1b5503547909ee9609fc.tgz
exp.interface.1307493.transactions.cor.2015-05-01.002417.tar.gz
transactions.cor.2015-05-01.002417.4f07864d29eb53dc8eb43f4de4f1e711.tgz
transactions.cor.2015-05-01.002417.chk.6f9bf302c7cf298565f1b8b131afa7f4.tgz
exp.interface.1308085.transactions.cor.2015-05-02.002419.tar.gz
transactions.cor.2015-05-02.002419.eeb2e0d6616c3cf443f77c34c2a27bd5.tgz
transactions.cor.2015-05-02.002419.chk.e3b894ff0eb31786d78a2774d4e7316c.tgz
exp.interface.1308789.transactions.cor.2015-05-02.002421.tar.gz
transactions.cor.2015-05-02.002421.7969fdfb132dabe6c200e2287f685cfe.tgz
transactions.cor.2015-05-02.002421.chk.2a8554aae7caed7c3887f90ca4fd5b8f.tgz
exp.interface.1309434.transactions.cor.2015-05-02.002423.tar.gz
transactions.cor.2015-05-02.002423.678ee6d0f05ce0f727108b0229ef14c2.tgz
transactions.cor.2015-05-02.002423.chk.d8bcb91a5eb5691face8d1f59a26ace5.tgz
exp.interface.1310048.transactions.cor.2015-05-02.002425.tar.gz
transactions.cor.2015-05-02.002425.f49937f7a93af6300ad93ed7bf57423a.tgz
transactions.cor.2015-05-02.002425.chk.d46740d8acc82b221dfa7b27389391c1.tgz
exp.interface.1310626.transactions.cor.2015-05-02.002427.tar.gz
transactions.cor.2015-05-02.002427.6527b1cab4530f8b91d6f6aa9cc37952.tgz
transactions.cor.2015-05-02.002427.chk.304d0babcccb8f293618d59fd8983884.tgz
exp.interface.1311208.transactions.cor.2015-05-02.002429.tar.gz
transactions.cor.2015-05-02.002429.b0ccb8dd11bb106b335915b8fb4985ce.tgz
transactions.cor.2015-05-02.002429.chk.2612b2f02abe419ea774efe75b1495b1.tgz
exp.interface.1311770.transactions.cor.2015-05-02.002431.tar.gz
transactions.cor.2015-05-02.002431.ca996b5cc8f29828ad62b40c390da738.tgz
transactions.cor.2015-05-02.002431.chk.a151e1837ed7ec4d530399bee61be07e.tgz
exp.interface.1312325.transactions.cor.2015-05-02.002433.tar.gz
transactions.cor.2015-05-02.002433.4ba12a47b9f44fe76a760a89d3e6f31e.tgz
transactions.cor.2015-05-02.002433.chk.398df00f78dd0049de322e01fe6746ca.tgz
exp.interface.1312919.transactions.cor.2015-05-03.002435.tar.gz
transactions.cor.2015-05-03.002435.09eeef9eb51974b1527078189c13ef8e.tgz
transactions.cor.2015-05-03.002435.chk.b506f65bf3385486d2170796f762a13e.tgz
exp.interface.1313624.transactions.cor.2015-05-03.002437.tar.gz
exp.interface.1314270.transactions.cor.2015-05-03.002439.tar.gz
transactions.cor.2015-05-03.002437.99f16075cba422ed1d94c74f8d07c2f4.tgz
transactions.cor.2015-05-03.002437.chk.ec7d341056ee26e7415eefa51f4e3288.tgz
transactions.cor.2015-05-03.002439.028a0e7f0721d31caa5441c42aa525de.tgz
transactions.cor.2015-05-03.002439.chk.a15749dec5b30b251e12623799f4dcac.tgz
exp.interface.1314880.transactions.cor.2015-05-03.002441.tar.gz
transactions.cor.2015-05-03.002441.3f550a3c8375bcf30e98cdc61569f608.tgz
transactions.cor.2015-05-03.002441.chk.839c3e5a4412a9dfd08c0066aae8689b.tgz
exp.interface.1315468.transactions.cor.2015-05-03.002443.tar.gz
transactions.cor.2015-05-03.002443.853d1ebc1aca469219eeac4703bbf79b.tgz
transactions.cor.2015-05-03.002443.chk.edad6e9122b540c65df8e8c750825456.tgz
exp.interface.1316054.transactions.cor.2015-05-03.002445.tar.gz
transactions.cor.2015-05-03.002445.5670aa3c41371b4b0fe92c0ab44327c1.tgz
transactions.cor.2015-05-03.002445.chk.bcce7f279b3ae9b4eba37876207ae182.tgz
exp.interface.1316608.transactions.cor.2015-05-03.002447.tar.gz
transactions.cor.2015-05-03.002447.5f9ef688d904286bb9b06af7ede4c0be.tgz
transactions.cor.2015-05-03.002447.chk.0295b8d5a628e23e4f3465a3bfcaa8d1.tgz
exp.interface.1317160.transactions.cor.2015-05-03.002449.tar.gz
transactions.cor.2015-05-03.002449.bcb75d6bd1b2215707f888f624805f55.tgz
transactions.cor.2015-05-03.002449.chk.56f92b5bd0d6380ab191c7e55a3db3e5.tgz
exp.interface.1317760.transactions.cor.2015-05-04.002451.tar.gz
transactions.cor.2015-05-04.002451.6edcbbb58ff42f989c56749ea8f86b80.tgz
transactions.cor.2015-05-04.002451.chk.af02de8f4f0f897ba8b736d766306eeb.tgz
exp.interface.1318464.transactions.cor.2015-05-04.002453.tar.gz
transactions.cor.2015-05-04.002453.a9a94fbf15fadd6a78744501c4a6ba49.tgz
transactions.cor.2015-05-04.002453.chk.f2b0cead4e5456bd86d962674268dfa8.tgz
exp.interface.1319107.transactions.cor.2015-05-04.002455.tar.gz
transactions.cor.2015-05-04.002455.4f43e8a048050da400731d329847f5d7.tgz
transactions.cor.2015-05-04.002455.chk.b9ccf5ad05f922e9d7636cfa7318106e.tgz
exp.interface.1319722.transactions.cor.2015-05-04.002457.tar.gz
transactions.cor.2015-05-04.002457.02aef18177c7ee7019ead0287b958461.tgz
transactions.cor.2015-05-04.002457.chk.16dcdd6e11f7dd0ebe90c08d660e0995.tgz
exp.interface.1320309.transactions.cor.2015-05-04.002459.tar.gz
transactions.cor.2015-05-04.002459.5f51a68060af20116c2930a3e174bf4f.tgz
transactions.cor.2015-05-04.002459.chk.53be659573fb8adbdb08ed2bf360c23b.tgz
exp.interface.1320888.transactions.cor.2015-05-04.002461.tar.gz
transactions.cor.2015-05-04.002461.7963565bd984bccc9cd6b44f95c4fd58.tgz
transactions.cor.2015-05-04.002461.chk.00b7543c2fdf50f8853abb8cd4d616c9.tgz
exp.interface.1321445.transactions.cor.2015-05-04.002463.tar.gz
transactions.cor.2015-05-04.002463.fb298cc433d4b6946d6bb1179c3352ff.tgz
transactions.cor.2015-05-04.002463.chk.3ac335da4046211e1afe88565e5290a1.tgz
exp.interface.1321993.transactions.cor.2015-05-04.002465.tar.gz
transactions.cor.2015-05-04.002465.127e5c7af469c167114b62ef562ce883.tgz
transactions.cor.2015-05-04.002465.chk.1eeb9366c85b2ee6efcc51c3da6c1fa2.tgz
exp.interface.1322596.transactions.cor.2015-05-05.002467.tar.gz
transactions.cor.2015-05-05.002467.41ae0e29651fba731c2501301d655efa.tgz
transactions.cor.2015-05-05.002467.chk.c6321b180a78a94f58d071fbc01e650c.tgz
exp.interface.1323303.transactions.cor.2015-05-05.002469.tar.gz
transactions.cor.2015-05-05.002469.ab4439f269c30c2764d50b74271d45f3.tgz
transactions.cor.2015-05-05.002469.chk.1e8a386198470edcd9cf43ed70a4168e.tgz
exp.interface.1323944.transactions.cor.2015-05-05.002471.tar.gz
transactions.cor.2015-05-05.002471.dd73123909967f907e5692347b01e11b.tgz
transactions.cor.2015-05-05.002471.chk.b3ed1a47580b50e61dfa08f11792f6b6.tgz
exp.interface.1324570.transactions.cor.2015-05-05.002473.tar.gz
transactions.cor.2015-05-05.002473.b81d210295ca5cd25289a68aa112b575.tgz
transactions.cor.2015-05-05.002473.chk.44369c78affd438bc3bfcc3e8a3fd80b.tgz
exp.interface.1325157.transactions.cor.2015-05-05.002475.tar.gz
transactions.cor.2015-05-05.002475.843a216425681d6778894b7812b684f2.tgz
transactions.cor.2015-05-05.002475.chk.153f1c4957fa8f093e59830a66e63473.tgz
exp.interface.1325738.transactions.cor.2015-05-05.002477.tar.gz
transactions.cor.2015-05-05.002477.1137cf1010af69a8b41f6a4ff6189f9a.tgz
transactions.cor.2015-05-05.002477.chk.5bb9e1dfaaf1ff24b73daf76339cd62c.tgz
exp.interface.1326288.transactions.cor.2015-05-05.002479.tar.gz
transactions.cor.2015-05-05.002479.8b088f749460b93a183799d417b0092c.tgz
transactions.cor.2015-05-05.002479.chk.a88dda5fcaa4fa33a59cfae969dc5e1c.tgz
exp.interface.1326834.transactions.cor.2015-05-05.002481.tar.gz
transactions.cor.2015-05-05.002481.0ee0bfaea345c3167ed7b30ff34a98d9.tgz
transactions.cor.2015-05-05.002481.chk.b9ae8b670e091cd75b3a6ce9c4b3c036.tgz
exp.interface.1327436.transactions.cor.2015-05-06.002483.tar.gz
transactions.cor.2015-05-06.002483.af6549f393d27dc78a7544f42d6dd570.tgz
transactions.cor.2015-05-06.002483.chk.26b3627a58b70d82bf666d9cbc4e7d98.tgz
exp.interface.1328144.transactions.cor.2015-05-06.002485.tar.gz
exp.interface.1328781.transactions.cor.2015-05-06.002487.tar.gz
transactions.cor.2015-05-06.002485.2bd23fb03fe6b48b4110a35b40da7dd0.tgz
transactions.cor.2015-05-06.002485.chk.222dcabc5ccfa6c0fe61b13be1ac3b06.tgz
transactions.cor.2015-05-06.002487.cec42a84aaf31048713bc83ffa78a2c3.tgz
transactions.cor.2015-05-06.002487.chk.417aea0ad4f733f005462785cf8b8aa3.tgz
exp.interface.1329384.transactions.cor.2015-05-06.002489.tar.gz
transactions.cor.2015-05-06.002489.9c4b19e84be4d1d1482cfbac473d7764.tgz
transactions.cor.2015-05-06.002489.chk.0f914c2e77efee534746acb03e576618.tgz
exp.interface.1329972.transactions.cor.2015-05-06.002491.tar.gz
transactions.cor.2015-05-06.002491.8b9db656a66629b47219f979418c6541.tgz
transactions.cor.2015-05-06.002491.chk.d9f756faef0c67bac6099d82f1a7b98c.tgz
exp.interface.1330551.transactions.cor.2015-05-06.002493.tar.gz
transactions.cor.2015-05-06.002493.13ef91ca2fada5ecc972011bfff6fbed.tgz
transactions.cor.2015-05-06.002493.chk.f8a0f747daafda74f5559fed9455d787.tgz
exp.interface.1331106.transactions.cor.2015-05-06.002495.tar.gz
transactions.cor.2015-05-06.002495.665139f339f29ff7fdcaaf2a5c56bcc9.tgz
transactions.cor.2015-05-06.002495.chk.033689678f3bffdd1c78be2064cb3681.tgz
exp.interface.1331666.transactions.cor.2015-05-06.002497.tar.gz
transactions.cor.2015-05-06.002497.a925363d76a3d97ac364544ae4118679.tgz
transactions.cor.2015-05-06.002497.chk.a2f855be7d6ad0cbde459154c7ad6f84.tgz
exp.interface.1332271.transactions.cor.2015-05-07.002499.tar.gz
transactions.cor.2015-05-07.002499.adafe2b73a0c5c653046672131dc021f.tgz
transactions.cor.2015-05-07.002499.chk.1109302d88f1da586925698df2d10a7d.tgz
exp.interface.1332978.transactions.cor.2015-05-07.002501.tar.gz
transactions.cor.2015-05-07.002501.7b557e32b07ae9a5105c365c8517f147.tgz
transactions.cor.2015-05-07.002501.chk.b62afb04835adac1e247007508a7ad37.tgz
exp.interface.1333623.transactions.cor.2015-05-07.002503.tar.gz
transactions.cor.2015-05-07.002503.7b1a24ce9847ede90c412d1c121b4144.tgz
transactions.cor.2015-05-07.002503.chk.854720b16cea034d14d0d52295631191.tgz
exp.interface.1334242.transactions.cor.2015-05-07.002505.tar.gz
transactions.cor.2015-05-07.002505.254ded0792208442326837f4219a3c51.tgz
transactions.cor.2015-05-07.002505.chk.d4c91b98bf4c7bce0a13cb5ca3637cd5.tgz
exp.interface.1334837.transactions.cor.2015-05-07.002507.tar.gz
transactions.cor.2015-05-07.002507.fc0a2fd466d34298334b9208453d371e.tgz
transactions.cor.2015-05-07.002507.chk.7e7df789b4fb9640df6cf29e55335059.tgz
exp.interface.1335410.transactions.cor.2015-05-07.002509.tar.gz
transactions.cor.2015-05-07.002509.23137badbc153e06945f0b1733c4ea20.tgz
transactions.cor.2015-05-07.002509.chk.18607fc2043a8ee2d86e9888157df0b5.tgz
exp.interface.1335960.transactions.cor.2015-05-07.002511.tar.gz
transactions.cor.2015-05-07.002511.945eb9dffe79e6be1852e3ded2e87a9a.tgz
.
.
.
...2015-10-31...
...2015-10-31...
...2015-10-31...
...2015-10-31...
...2015-10-31...
I can straight a way ignore the files starting with
exp and
transactions files which contains
.chk.
In rest of all other files starting from
...2015-05-01... to
...2015-10-31... , i need the count of string (or number) 505001 and 602001 and display on date wise. here i am using the shell script, but some where i am missing the logic.
Code :
#!/bin/bash
count_505001=0
count_602001=0
tempdate=${file:-"2015-05-01"}
for file in transactions.cor.2015-0[56789]; do
if [[ $tempdate = ${file:18:10} ]]; then
tempdate=${file:18:10}
count_505001=count+`zless $file | egrep "505001" | wc -l`;
count_602001=count+`zless $file | egrep "602001" | wc -l`;
else
echo "Count for 505001 :"$count_505001" on date: "$tempdate
echo "Count for 602001 :"$count_602001" on date: "$tempdate
tempdate=${file:18:10}
count_505001=`zless $file | egrep "505001" | wc -l`;
count_602001=`zless $file | egrep "602001" | wc -l`;
fi
done;
here is the example output:
Code :
Count for 505001 :236 on date: 2015-05-01
Count for 602001 :371 on date: 2015-05-01
Count for 505001 :9 on date: 2015-05-02
Count for 602001 :42 on date: 2015-05-02
Count for 505001 :53 on date: 2015-05-03
Count for 602001 :63 on date: 2015-05-03
.
.
.
Count for 505001 :423 on date: 2015-10-31
Count for 602001 :745 on date: 2015-10-31
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am new to shell programming. Can anyone help me out with anyone of these?
Display a count of the number of regular files, the number of symbolic links, the number of sub-directories, the number of block-special files, and the number of character-special files in the directory.
I don't... (4 Replies)
Discussion started by: wayne1411
4 Replies
2. Shell Programming and Scripting
I want a shell script which extract data from a log file which contains date and time-wise data and i need the data for a perticular interval of time...what can i do??? (3 Replies)
Discussion started by: abhishek27
3 Replies
3. Shell Programming and Scripting
Hi all,
this is my first post, i need to write a script to zip the files with datewise below are the log files.
-rw------- 1 root sso 85316156 May 24 22:11 core_test_smservaz_104_104_1243217459_8896
-rw------- 1 root sso 90413304 May 25 22:12 core_test_smservaz_104_104_1243303895_20912... (4 Replies)
Discussion started by: lcschandu
4 Replies
4. Solaris
Hi guys,
I want to delete files from june 13 to june 30, using rm command can any one tell me the sintax to remove. I ahve hunderd of core files in my /var dir. so i want to clear last month core files. Thanks in Advance.:)) (2 Replies)
Discussion started by: kurva
2 Replies
5. Shell Programming and Scripting
Hi,
I search all forum, but I can not find solutions of my problem :(
I have multiple files (5000 files), inside there is this data :
FILE 1:
1195.921 -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05
-2021.287 1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05
... (15 Replies)
Discussion started by: guns
15 Replies
6. Shell Programming and Scripting
Hi All,
I would have many files in the server with
xyz*.dat -- Static file name
Physical files:
xyz1.dat - 01PM
xyz2.dat - 02PM
xyz3.dat - 03PM
In present version we are using
for f in $file_name
do
fname=`ls $f | grep -v ^'\|'$ | sed s/' '/'\\ '/g`
....
sqlldr... (4 Replies)
Discussion started by: Dharv
4 Replies
7. UNIX for Dummies Questions & Answers
My unix version is IBM AIX Version 6.1
I tried google my requirement and found the below answer,
find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20"
But newer command is not working in AIX version 6.1 unix
I have given my requirement below:
Input:
atr files:
... (1 Reply)
Discussion started by: yuvaa27
1 Replies
8. Shell Programming and Scripting
Hi,
Very good wishes to all!
Please help to provide the shell script for generating the record counts in filed wise from the .csv file
My question:
Source file:
Field1 Field2 Field3
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn
My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies
9. Shell Programming and Scripting
Hello Friends,
Can somebody assist an issue I am having? I have a separate file with a list of account ids
XXX200B02Y01
XXX200B03Y01
XXX200B05Y01
XXX200B07Y01
XXX200B08Y01
I call the file, and run an egrep against a directory and logfiles
AccountID=$(cat... (2 Replies)
Discussion started by: liketheshell
2 Replies
10. Shell Programming and Scripting
I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number
File1
Name = Joe
Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies