Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extract count of string in all files and display on date wise
Top Forums Shell Programming and Scripting Extract count of string in all files and display on date wise Post 302959927 by Don Cragun on Sunday 8th of November 2015 01:10:31 AM
Old 11-08-2015
It would seem that the following script should do what you were trying to do:
Code:
#!/bin/bash
shopt -s nullglob
for file in transactions.cor.2015-0[56789]*
do	[[ "$file" != ${file##*.chk.} ]] && continue
	if [[ $tempdate = ${file:17:10} ]]
	then	count_505001=$((count_505001 + $(zless "$file" | fgrep -c 505001)))
		count_602001=$((count_602001 + $(zless "$file" | fgrep -c 602001)))
		continue
	fi
	if [[ $tempdate != "" ]]
	then	echo "Count for 505001: $count_505001 on date: $tempdate"
		echo "Count for 602001: $count_602001 on date: $tempdate"
	fi
	tempdate=${file:17:10}
	count_505001=$(zless "$file" | fgrep -c 505001)
	count_602001=$(zless "$file" | fgrep -c 602001)
done
if [[ $tempdate != "" ]]
then	echo "Count for 505001: $count_505001 on date: $tempdate"
	echo "Count for 602001: $count_602001 on date: $tempdate"
fi

Note that several syntax errors have been corrected, it now skips *.chk.* files, it prints results from the sum of the counts for each date (instead of the count from the last file on each date), and it prints results from the last date processed.

Note also that fgrep is usually faster than egrep and fgrep -c is definitely faster than egrep | wc -l.

This could all be built into an awk script that would only need to zless each file once (instead of twice) and could eliminate the grep (and wc) invocations completely.


Late fixes added:
  1. Changed [[ template != "" ]] to [[ $template != "" ]] in two places.
  2. Added shopt -s nullglob.
Last edited by Don Cragun; 11-08-2015 at 06:19 PM.. Reason: Add late fixes...
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Display the count of files

I am new to shell programming. Can anyone help me out with anyone of these? Display a count of the number of regular files, the number of symbolic links, the number of sub-directories, the number of block-special files, and the number of character-special files in the directory. I don't... (4 Replies)
Discussion started by: wayne1411
4 Replies

2. Shell Programming and Scripting

To extract data of a perticular interval (date-time wise)

I want a shell script which extract data from a log file which contains date and time-wise data and i need the data for a perticular interval of time...what can i do??? (3 Replies)
Discussion started by: abhishek27
3 Replies

3. Shell Programming and Scripting

Need to zip the files date wise --urgent Help needed

Hi all, this is my first post, i need to write a script to zip the files with datewise below are the log files. -rw------- 1 root sso 85316156 May 24 22:11 core_test_smservaz_104_104_1243217459_8896 -rw------- 1 root sso 90413304 May 25 22:12 core_test_smservaz_104_104_1243303895_20912... (4 Replies)
Discussion started by: lcschandu
4 Replies

4. Solaris

delete files by date wise

Hi guys, I want to delete files from june 13 to june 30, using rm command can any one tell me the sintax to remove. I ahve hunderd of core files in my /var dir. so i want to clear last month core files. Thanks in Advance.:)) (2 Replies)
Discussion started by: kurva
2 Replies

5. Shell Programming and Scripting

Extract string from multiple file based on line count number

Hi, I search all forum, but I can not find solutions of my problem :( I have multiple files (5000 files), inside there is this data : FILE 1: 1195.921 -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05 -2021.287 1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05 ... (15 Replies)
Discussion started by: guns
15 Replies

6. Shell Programming and Scripting

How to read files by Server Creation date wise?

Hi All, I would have many files in the server with xyz*.dat -- Static file name Physical files: xyz1.dat - 01PM xyz2.dat - 02PM xyz3.dat - 03PM In present version we are using for f in $file_name do fname=`ls $f | grep -v ^'\|'$ | sed s/' '/'\\ '/g` .... sqlldr... (4 Replies)
Discussion started by: Dharv
4 Replies

7. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

8. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

9. Shell Programming and Scripting

Count the number of string occurrences to display 0 entries in output

Hello Friends, Can somebody assist an issue I am having? I have a separate file with a list of account ids XXX200B02Y01 XXX200B03Y01 XXX200B05Y01 XXX200B07Y01 XXX200B08Y01 I call the file, and run an egrep against a directory and logfiles AccountID=$(cat... (2 Replies)
Discussion started by: liketheshell
2 Replies

10. Shell Programming and Scripting

Search string in multiple files and display column wise

I have 3 files. Each of those files have the same number of records, however certain records have different values. I would like to grep the field in ALL 3 files and display the output with only the differences in column wise and if possible line number File1 Name = Joe Age = 33... (3 Replies)
Discussion started by: sidnow
3 Replies

LEARN ABOUT FREEBSD

zless

ZMORE(1)						    BSD General Commands Manual 						  ZMORE(1)

NAME

     zmore, zless -- view compressed files

SYNOPSIS

     zmore [flags] [file ...]
     zless [flags] [file ...]

DESCRIPTION

     zmore is a filter that allows the viewing of files compressed with Lempel-Ziv encoding.  Such files generally have a ``Z'' or ``gz'' exten-
     sion (both the compress(1) and gzip(1) formats are supported).  Any flags that are specified are passed to the user's preferred PAGER (which
     is /usr/bin/more by default).

     zless is equivalent to zmore but uses less(1) as a pager instead of more(1).

     When multiple files are specified, zmore will pause at the end of each file and present the following prompt to the user:

	   prev_file (END) - Next: next_file

     Where prev_file is the file that was just displayed and next_file is the next file to be displayed.  The following keys are recognized at the
     prompt:

	   e or q  quit zmore.

	   s	   skip the next file (or exit if the next file is the last).

     If no files are specified, zmore will read from the standard input.  In this mode zmore will assume gzip(1) style compression since there is
     no suffix on which to make a decision.

ENVIRONMENT

     PAGER  Program used to display files.  If unset, /usr/bin/more is used (zmore) or /usr/bin/less (zless).

SEE ALSO

     compress(1), less(1), more(1)

BSD
								 October 22, 2014							       BSD
All times are GMT -4. The time now is 11:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy