How to find repeated string in a text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to find repeated string in a text file
# 8  
Old 10-25-2011
Code:
nawk '{c+=gsub("ST[*]850[*]", "&")}END{print c}' myFile

# 9  
Old 10-25-2011
rdcwayx
it works fine, but now is more complex, so I have a Directory with 70,000 files with differents dates, so the goal is for example get how many times the string is repeated in all files from a date ( for example the September month)
Thanks for your help.
# 10  
Old 10-25-2011
Quote:
Originally Posted by cucosss
rdcwayx
it works fine, but now is more complex, so I have a Directory with 70,000 files with differents dates, so the goal is for example get how many times the string is repeated in all files from a date ( for example the September month)
Thanks for your help.
take a look at my previous suggestion - substitute the 'myFile' with the wild-carded file names for month of September. I we don't know how your files are named, it's hard to provide a more detailed hint.
The below will 'grab' all the files and provide the total. Start with that.
Code:
nawk '{c+=gsub("ST[*]850[*]", "&")}END{print c}' /path/2/dir/with/files/*

This User Gave Thanks to vgersh99 For This Post:
# 11  
Old 10-25-2011
Quote:
Originally Posted by cucosss
rdcwayx
it works fine, but now is more complex, so I have a Directory with 70,000 files with differents dates, so the goal is for example get how many times the string is repeated in all files from a date ( for example the September month)
Thanks for your help.
Here is the update for your new request
Code:
awk -F "ST\\\*850\\\*" '{sum+=NF-1}END{print sum}' *sept*


Last edited by rdcwayx; 10-25-2011 at 07:27 PM..
# 12  
Old 10-25-2011
ok guys, Im trying this scritp:

clear
echo "Please enter the start date in the format MMM DD, example: Jun 25"
read strtdt
echo "please enter the end date in the format MMM DD, example: Ago 26"
read enddt
touch -t ${strtdt}0000 /gentran/SI51/install/EDIS_Inbound/datefrom
touch -t ${enddt}2359 /gentran/SI51/install/EDIS_Inbound/dateto
find /gentran/SI51/install/EDIS_Inbound \( -newer /gentran/SI51/install/EDIS_Inb
ound/datefrom ! -newer /gentran/SI51/install/EDIS_Inbound/dateto \) -print | xar
gs awk '{c+=gsub("ST[*]850[*]", "&")}END{print c}'

but when the dates are small, for instead 20111024 to 20111025 it appears work fine, but when dates are bigger for instead 20111001 to 20111025 then I get several lines like: 1031
691
463
98
132
148
16
Do you know why?
# 13  
Old 10-26-2011
xargs causes awk to run several times. Change "xargs awk" to "xargs cat |awk". This way xargs invokes cat as many times as needed then all the concatenated data is fed into one awk.
This User Gave Thanks to binlib For This Post:
# 14  
Old 10-26-2011
All.
Thanks so much for your help, I have gotten the result I wanted, binlib, your help was very useful also.

Refugio.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Find and replace a string in a text file

Dear all, I want to find all the "," in my text file and then replace the commas to a tab. I found a script online but I don't know how to modify the script for my case. Any one can help? Thank you. @echo off &setlocal set "search=%1" set "replace=%2" set "textfile=Input.txt" set... (2 Replies)
Discussion started by: forevertl
2 Replies

2. Shell Programming and Scripting

[Need help] perl script to find the occurance of string from a text file

I have two files 1. input.txt 2. keyword.txt input.txt has contents like .src_ref 0 "call.s" 24 first 0x000000 0x5a80 0x0060 BRA.l 0x60 .src_ref 0 "call.s" 30 first 0x000002 0x1bc5 RETI .src_ref 0 "call.s" 31 first 0x000003 0x6840 ... (2 Replies)
Discussion started by: acdc
2 Replies

3. Shell Programming and Scripting

Find repeated word and take sum of the second field to it ,for all the repeated words in awk

Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it Input fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Discussion started by: 100bees
11 Replies

4. Shell Programming and Scripting

find string(s) in text file and nearby data, export to list help

Hi, So I'm kinda new to shell scripts and the like. I've picked up quite a bit of use from browsing the forums here but ran into a new one that I can't seem to find an answer for. I'm looking to parse/find a string AND the next 15 or so charachters that follow the string within a text file... (1 Reply)
Discussion started by: kar23me
1 Replies

5. Shell Programming and Scripting

Find string in text file

Hello! Please, help me to write such script. I have some text file with name filename.txt I must check if this file contains string "test-string-first", I must cut from this file string which follows string "keyword-string:" and till first white-space and save it to some variable. For... (3 Replies)
Discussion started by: optik77
3 Replies

6. Shell Programming and Scripting

find a string in a file and add some text after that file

Hi Could you please help me out by solving teh below problem ? I have a file with as below source1|target1|yes source2|target2|no source1 is file in which i have to place some code under the <head> tag in it. What code i have to place in source1 is something like this "abcd.....<target1>... (5 Replies)
Discussion started by: Tasha_T
5 Replies

7. Shell Programming and Scripting

Extract multiple repeated data from a text file

Hi, I need to extract data from a text file in which data has a pattern. I need to extract all repeated pattern and then save it to different files. example: input is: ST*867*000352214 BPT*00*1000352214*090311 SE*1*1 ST*867*000352215 BPT*00*1000352214*090311 SE*1*2 ... (5 Replies)
Discussion started by: apjneeraj
5 Replies

8. Shell Programming and Scripting

Delete repeated word in text file

Hi expert, I am using C shell. And i trying to delete repeated word. Example file.txt: BLUE YELLOW RED VIOLET RED RED BLUE WHITE YELLOW BLACK and i wan store the output into a new file: BLUE (6 Replies)
Discussion started by: vincyoxy
6 Replies

9. Shell Programming and Scripting

Looking for command(s)/ script to find a text string within a file

I need to search through all files with different file suffixes in a directory structure to locate any files containing a specific string (5 Replies)
Discussion started by: wrwelden
5 Replies

10. UNIX for Dummies Questions & Answers

how to find a word repeated in a file

Hi everyone, I have a file in which a word is repeated more than one time and I want to know how many times it is repeated. ex: if i repeated word 'guru' in 10 lines I can get the o/p as: cat filename | grep -c 'guru'. How ever if the word is repeated more than one time, then how can I... (4 Replies)
Discussion started by: gurukottur
4 Replies
Login or Register to Ask a Question