Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help in locating a word in huge files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help in locating a word in huge files
# 1  
Old 04-25-2010
MySQL Help in locating a word in huge files
hi

i receive about 5000 files per day in my system. Each of them are like:

cat ABC.april24.dat
ABH00001990 01993 409009092 0909 INI iop 9033
AAB0000237893784 8430900 898383 AUS 34349089008 849843 9474822
AAA00003849893498098394 84834 348348439 -438939 IN
AAA00004438493893849384 89809 978980980 -898988 IND
.
.
.

each of these files have millions of line like this, and Each line has fix format depending upon category (ABH, AAB,AAA), i want to find all AAA category in the file which have IN any where in location 54-60, as this is invalid value and IND is correct. So like this i want to identiy all AAA lines which has invalid value 'IN' instead of 'IND'.

finding it difficult due to huge size of these files

Thanks in advance !!!
# 2  
Old 04-26-2010
Not shure what output you desire but you could do something like:
Code:
awk '/^AAA/ && $NF=="IN$"{print "File: "FILENAME: ", line "FNR}' files...

Last edited by Franklin52; 04-26-2010 at 04:03 AM.. Reason: adding condition : /^AAA/ &&
# 3  
Old 04-26-2010
If the lines begin with "AAA" and end with "IN" followed by optional spaces:
Code:
$ sed -n '/^AAA.*IN *$/p' ABC.april24.dat 
AAA00003849893498098394 84834 348348439 -438939 IN

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know... (1 Reply)
Discussion started by: Prathi
1 Replies

2. Shell Programming and Scripting

Find a word and increment the number in the word & save into new files

Hi All, I am looking for a perl/awk/sed command to auto-increment the numbers line in file, P1.tcl: run_build_model sparc_ifu_dec run_drc set_faults -model path_delay -atpg_effectiveness -fault_coverage add_delay_paths P1 set_atpg -abort_limit 1000 run_atpg -ndetects 1000 I would like... (6 Replies)
Discussion started by: jypark22
6 Replies

3. UNIX for Dummies Questions & Answers

Locating shell script files with the $PATH variable

I've created a test script, which is located in $HOME/bin. The script runs as expected with no issues. However, upon echo'ing the $path variable the location of my script is not located in any of the directories listed in $path. So my question is, how does shell know where the script is located... (2 Replies)
Discussion started by: BrandonD
2 Replies

4. Solaris

Locating files!

My question is : -How do I locate files of specific user in specific folder. In other words, How do I find files that belong only to "root" user, in /etc folder? (5 Replies)
Discussion started by: alexex666
5 Replies

5. UNIX for Dummies Questions & Answers

Find EXACT word in files, just the word: no prefix, no suffix, no 'similar', just the word

I have a file that has the words I want to find in other files (but lets say I just want to find my words in a single file). Those words are IDs, so if my word is ZZZ4, outputs like aaZZZ4, ZZZ4bb, aaZZZ4bb, ZZ4, ZZZ, ZyZ4, ZZZ4.8 (or anything like that) WON'T BE USEFUL. I need the whole word... (6 Replies)
Discussion started by: chicchan
6 Replies

6. Shell Programming and Scripting

Find and replace a word in all the files (that contain the word) under a directory

Hi Everyone, I am looking for a simple way for replacing all the files under a directory that use the server "xsgd1234dap" with "xsdr3423pap". For Example: In the Directory, $pwd /home/nick $ grep -l "xsgd1234dap" *.sh | wc -l 119 I have "119" files that are still using... (5 Replies)
Discussion started by: filter
5 Replies

7. UNIX for Dummies Questions & Answers

Script to search for a particular word in files and print the word and path name

Hi, i am new to unix shell scripting and i need a script which would search for a particular word in all the files present in a directory. The output should have the word and file path name. For example: "word" "path name". Thanks for the reply in adv,:) (3 Replies)
Discussion started by: virtual_45
3 Replies

8. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

9. UNIX for Advanced & Expert Users

Huge files manipulation

Hi , i need a fast way to delete duplicates entrys from very huge files ( >2 Gbs ) , these files are in plain text. I tried all the usual methods ( awk / sort /uniq / sed /grep .. ) but it always ended with the same result (memory core dump) In using HP-UX large servers. Any advice will... (8 Replies)
Discussion started by: Klashxx
8 Replies

10. Filesystems, Disks and Memory

Locating files and directing ouput

I'm trying to locate all files that do not contain the string /usr but do contain the string csh within the file called /etc/passwd. Then I would like to direct this output to a file called pout. Does anyone one have suggestions on this one? Thanks (2 Replies)
Discussion started by: klannon
2 Replies
Login or Register to Ask a Question