Sponsored Content
Full Discussion: Optimizing search using grep
Top Forums Shell Programming and Scripting Optimizing search using grep Post 302965557 by Junaid Subhani on Saturday 30th of January 2016 08:57:28 PM
Old 01-30-2016
Optimizing search using grep

I have a huge log file close to 3GB in size.

My task is to generate some reporting based on # of times something is being logged.

I need to find the number of time StringA , StringB , StringC is being called separately.

What I am doing right now is:

Code:
grep "StringA" server.log | wc -l
grep "StringB" server.log | wc -l
grep "StringC" server.log | wc -l

This is a long process and my script takes close to 10 minutes to complete. What I want to know is that whether this can be optimized or not ? Is is possible to run one grep command and find out the number of time StringA, StringB and StringC has been called individually ?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

search pattern by grep

hai folks, i am vijay very new to this website. My query: Search patterns from root directory to all directories by using grep (3 Replies)
Discussion started by: vijaysabari
3 Replies

2. Shell Programming and Scripting

how to search -n in a file with grep

can anybody let me know how can i search for "-n" in a text file; or if my file name is "-n" (which is legal in Unix) how do i deal with that? thanks (2 Replies)
Discussion started by: lamilami
2 Replies

3. Shell Programming and Scripting

search using grep

Hi all, I have a folder with many files. I need to get all the filenames with the value "<f id=audio-time>*:*:*</f>" word. * may be any integer. I used find . -type f -exec grep -H '<f id=audio-time>.:.</f>' {} \; command to get. but i am not able to find the correct values. Any... (2 Replies)
Discussion started by: ananthi_ku
2 Replies

4. UNIX for Dummies Questions & Answers

Speeding/Optimizing GREP search on CSV files

Hi all, I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min). Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search. Example:... (11 Replies)
Discussion started by: Whit3H0rse
11 Replies

5. Shell Programming and Scripting

Grep Search for both First and last word

Hi Pretty sure this is very simple hence im banging my head o the wall as to why i cant get this to work: Im greping against many files where there will be the following string: Date: Thu, Aug 23 2001 09:27 PM (of course values such as date ,time and PM/AM will vary.) Im trying to grep... (1 Reply)
Discussion started by: duonut
1 Replies

6. Shell Programming and Scripting

search for a pattern using grep

Hi I am facing the below problem. I have set of lines in which i have to search for only the line which matches with the pattren "/" only. input:- /*+ some text */ /*+ some text */ /* Remove rows from a table of survey results. */ /* Add a survey respondent's name and answers. */ /*... (7 Replies)
Discussion started by: manasa_vs
7 Replies

7. Shell Programming and Scripting

Re: grep exact search

Hi, x=GATE ps -ef|grep pmon grid 552 1 0 Aug08 ? 00:00:51 asm_pmon_+ASM1 oracle 4314 1 0 Aug08 ? 00:04:08 ora_pmon_GATE1 oracle 5018 1351 0 10:14 pts/1 00:00:00 grep pmon oracle 7329 1 0 Aug08 ? 00:02:19 ora_pmon_NRID1 ps... (3 Replies)
Discussion started by: rcc50886
3 Replies

8. Shell Programming and Scripting

Grep for search.

hi, i got files delimited by |^ character. eg. apple|^ball|^cat|^dog|^egg i need to find if the above file contains | (pipe ) character inside the data.eg. apple|^ball|^cat|^d|og|^egg. above file contains | character in the dog data. how can i search that. (4 Replies)
Discussion started by: krk
4 Replies

9. UNIX for Beginners Questions & Answers

Grep/awk using a begin search pattern and end search pattern

I have this fileA TEST FILE ABC this file contains ABC; TEST FILE DGHT this file contains DGHT; TEST FILE 123 this file contains ABC, this file contains DEF, this file contains XYZ, this file contains KLM ; I want to have a fileZ that has only (begin search pattern for will be... (2 Replies)
Discussion started by: vbabz
2 Replies

10. UNIX for Beginners Questions & Answers

How to use a grep search to search for a specific string within multiple directories?

Lets say I have a massive directory which is filled with other directories all filled with different c++ scripts and I want a listing of all the scripts that contain the string: "this string". Is there a way to use a grep search for that? I tried: grep -lr "this string" * but I do not... (3 Replies)
Discussion started by: Circuits
3 Replies
CG(1)																	     CG(1)

NAME
cg - Recursively grep for a pattern and store it. SYNOPSIS
cg [ -l ] | [ [ -i ] pattern [ files ] ] DESCRIPTION
cg does a search though text files (usually source code) recursively for a pattern, storing matches and displaying the output in a human- readable fashion. It is intended to give some of the functionaly of AT&T's cscope(1) tool, with the advantages of simplicity and not being language-specific. The script will colorize output if configured as such. It is typically run with a Perl regular expression to search for. The search can be made case insensitive by using the -i option. A list of files may also be specified with an additional argument after the pattern. Put the files pattern in quotes to make it be matched by Perl rather than by the shell. Running the script with no arguments will recall the results of the previous search. After the search, entries found can be edited using the vg(1) script. The -l option shows the last log made. SOME EXAMPLES
cg - alone recalls the previous search results. cg -i pattern - search the default list of files for all files matching the pattern (and case-insensitively). cg pattern '*.c' - search recursively for pattern in all *.c files. This automatically converts '*' to '.*' and '.' to '.' for you and does a Perl pattern match on all files in the tree. cg pattern *.c - search through the shell-expanded list of *.c files, so not done recursively (in other words, only the files your shell pass to the script as arguments). cg -l - show the last log made. COMMAND-LINE OPTIONS -i Do a case-insensitive search. -l Show the last log made. -p Toggle the default pager option. cg has a bulit-in pager function, which can be enabled or disabled by default (in .cgvgrc). If the default is enabled, this option disables the pager; if the default is disabled, this option enables it. -P Force the built-in pager to be disabled. FILES
${HOME}/.cglast Log file of the last search. ${HOME}/.cgvgrc Per-user configuration file (if the defaults are not desireable). ${HOME}/.cgvg/* Log files in $HOSTNAME.shell_pid form with the log of the last search. SEE ALSO
vg(1), perl(1), find(1), grep(1), cscope(1) AUTHOR
cg was written by Joshua Uziel <uzi@uzix.org>. 13 Mar 2002 CG(1)
All times are GMT -4. The time now is 04:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy