Extract count of string in all files and display on date wise


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract count of string in all files and display on date wise
# 8  
Old 11-08-2015
Quote:
Originally Posted by Don Cragun
[...]
Code:
#!/bin/bash
for file in transactions.cor.2015-0[56789]*
do	[[ "$file" != ${file##*.chk.} ]] && continue
	if [[ $tempdate = ${file:17:10} ]]
	then	count_505001=$((count_505001 + $(zless "$file" | fgrep -c 505001)))
		count_602001=$((count_602001 + $(zless "$file" | fgrep -c 602001)))
		continue
	fi
	if [[ tempdate != "" ]]
	then	echo "Count for 505001: $count_505001 on date: $tempdate"
		echo "Count for 602001: $count_602001 on date: $tempdate"
	fi
	tempdate=${file:17:10}
	count_505001=$(zless "$file" | fgrep -c 505001)
	count_602001=$(zless "$file" | fgrep -c 602001)
done
if [[ tempdate != "" ]]
then	echo "Count for 505001: $count_505001 on date: $tempdate"
	echo "Count for 602001: $count_602001 on date: $tempdate"
fi

Hi Don,

[[ tempdate != "" ]] will always be true since the string tempdate is never "".
# 9  
Old 11-08-2015
Quote:
Originally Posted by Aia
Code:
#!/bin/bash
for file in transactions.cor.2015-0[56789]*
do	[[ "$file" != ${file##*.chk.} ]] && continue
	if [[ $tempdate = ${file:17:10} ]]
	then	count_505001=$((count_505001 + $(zless "$file" | fgrep -c 505001)))
		count_602001=$((count_602001 + $(zless "$file" | fgrep -c 602001)))
		continue
	fi
	if [[ tempdate != "" ]]
	then	echo "Count for 505001: $count_505001 on date: $tempdate"
		echo "Count for 602001: $count_602001 on date: $tempdate"
	fi
	tempdate=${file:17:10}
	count_505001=$(zless "$file" | fgrep -c 505001)
	count_602001=$(zless "$file" | fgrep -c 602001)
done
if [[ tempdate != "" ]]
then	echo "Count for 505001: $count_505001 on date: $tempdate"
	echo "Count for 602001: $count_602001 on date: $tempdate"
fi

Hi Don,
[[ tempdate != "" ]] will always be true since the string tempdate is never "".
Hi Aia,
$tempdate will expand to an empty string inside the for loop the first time a filename matching transactions.cor.2015-0[56789]* is found that does not contain the string .chk..

$tempdate will expand to an empty string after the for loop completes if, and only if, no filenames matching transactions.cor.2015-0[56789]* were found that did not contain the string .chk., or no filenames matching transactions.cor.2015-0[56789]* were found.

There should also be a check to be sure that at least one file matched the pattern transactions.cor.2015-0[56789]* (or the bash option that causes a non-matching pattern to expand to an empty list should be specified). I normally use ksh instead of bash. With ksh, I would avoid this problem by changing:
Code:
for file in transactions.cor.2015-0[56789]*

to:
Code:
for file in ~(N:transactions.cor.2015-0[56789]*)

which will not execute the loop if there are no matches. I always forget the option that enables this feature in bash and have to look it up, but for recent versions of bash, it is:
Code:
shopt -s nullglob
for file in transactions.cor.2015-0[56789]*

# 10  
Old 11-08-2015
Hi Don,

Then, shouldn't be: if [[ $tempdate != "" ]] ?
# 11  
Old 11-08-2015
Quote:
Originally Posted by Aia
Hi Don,

Then, shouldn't be: if [[ $tempdate != "" ]] ?
With single square brackets, you need the <dollar-sign> and quotes:
Code:
if [ "$template" != "" ]

because the shell is parsing operands to a built-in. But, double square brackets are part of the shell command language, not a built-in, so the <dollar-sign> and quotes are optional. All of the following produce the same results:
Code:
if [[ template != "" ]]
if [[ $template != "" ]]
if [[ "$template" != "" ]]

taking the then branch if $template expands to a non-empty string. But the following are equivalent to each other, but, unlike the above three commands, always take the then branch:
Code:
if [[ "template" != "" ]]
if [[ 'template' != "" ]]

# 12  
Old 11-08-2015
Quote:
Originally Posted by Don Cragun
[...] But, double square brackets are part of the shell command language, not a built-in, so the <dollar-sign> and quotes are optional. All of the following produce the same results:
Code:
if [[ template != "" ]]
if [[ $template != "" ]]
if [[ "$template" != "" ]]

Hi Don,
First, thank you for taking the time.

I understand that the properties of the double [[ does not suffer of the same issues that the single [ or test. It does not do word splitting nor globing, therefore you do not need to double quote the left-hand side of it and you only quote the right side if you do not want a pattern match, instead of a string comparison.

However, if [[ template != "" ]] does not expand since it is a literal string. How can it be the same that $template with or without quotes?

Perhaps, I can demonstrate my question better with a test.
Code:
$ cat test_test
#!/bin/bash
set -x
template=""
[[ template != "" ]] && echo "success!" || echo "failure!"
[[ $template != "" ]] && echo "success!" || echo "failure!"

Code:
$ ./test_test
+ template=
+ [[ template != '' ]] <-- it is the literal string template
+ echo 'success!'
success! <-- this should be failure! if it were dereferencing 
+ [[ '' != '' ]] <-- dereferenced to the value in template
+ echo 'failure!'
failure!


Last edited by Aia; 11-08-2015 at 05:14 PM..
This User Gave Thanks to Aia For This Post:
# 13  
Old 11-08-2015
Hi.

Arithmetic expressions work that way:
Code:
Within an expression,
       shell variables may also be referenced by name without using the
       parameter expansion syntax.

-- man bash, ARITHMETIC EVALUATION

Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
# 14  
Old 11-08-2015
Quote:
Originally Posted by drl
[...]

Arithmetic expressions work that way:
[...]
Hi drl,

Could you tell me when this if [[ template != "" ]] became an arithmetic expression?
As far as I understand it it is a string conditional expression. Maybe that's what I am missing. But if this were the case, then I would have to still believe that:
Quote:
Code:
if [[ template != "" ]]
if [[ $template != "" ]]
if [[ "$template" != "" ]]

All of the following produce the same results
it is not so?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search string in multiple files and display column wise

I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number File1 Name = Joe Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies

2. Shell Programming and Scripting

Count the number of string occurrences to display 0 entries in output

Hello Friends, Can somebody assist an issue I am having? I have a separate file with a list of account ids XXX200B02Y01 XXX200B03Y01 XXX200B05Y01 XXX200B07Y01 XXX200B08Y01 I call the file, and run an egrep against a directory and logfiles AccountID=$(cat... (2 Replies)
Discussion started by: liketheshell
2 Replies

3. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

4. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

5. Shell Programming and Scripting

How to read files by Server Creation date wise?

Hi All, I would have many files in the server with xyz*.dat -- Static file name Physical files: xyz1.dat - 01PM xyz2.dat - 02PM xyz3.dat - 03PM In present version we are using for f in $file_name do fname=`ls $f | grep -v ^'\|'$ | sed s/' '/'\\ '/g` .... sqlldr... (4 Replies)
Discussion started by: Dharv
4 Replies

6. Shell Programming and Scripting

Extract string from multiple file based on line count number

Hi, I search all forum, but I can not find solutions of my problem :( I have multiple files (5000 files), inside there is this data : FILE 1: 1195.921 -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05 -2021.287 1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05 ... (15 Replies)
Discussion started by: guns
15 Replies

7. Solaris

delete files by date wise

Hi guys, I want to delete files from june 13 to june 30, using rm command can any one tell me the sintax to remove. I ahve hunderd of core files in my /var dir. so i want to clear last month core files. Thanks in Advance.:)) (2 Replies)
Discussion started by: kurva
2 Replies

8. Shell Programming and Scripting

Need to zip the files date wise --urgent Help needed

Hi all, this is my first post, i need to write a script to zip the files with datewise below are the log files. -rw------- 1 root sso 85316156 May 24 22:11 core_test_smservaz_104_104_1243217459_8896 -rw------- 1 root sso 90413304 May 25 22:12 core_test_smservaz_104_104_1243303895_20912... (4 Replies)
Discussion started by: lcschandu
4 Replies

9. Shell Programming and Scripting

To extract data of a perticular interval (date-time wise)

I want a shell script which extract data from a log file which contains date and time-wise data and i need the data for a perticular interval of time...what can i do??? (3 Replies)
Discussion started by: abhishek27
3 Replies

10. Shell Programming and Scripting

Display the count of files

I am new to shell programming. Can anyone help me out with anyone of these? Display a count of the number of regular files, the number of symbolic links, the number of sub-directories, the number of block-special files, and the number of character-special files in the directory. I don't... (4 Replies)
Discussion started by: wayne1411
4 Replies
Login or Register to Ask a Question