Text search


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Text search
# 1  
Old 02-25-2011
Text search

Hello all,

I want to find all the words that comes after certain words, I need to do that in order to process some log files for example

Suppose that, I have this text

Hello, I am from country_A, I am living in country_B.

Now, I want to get all the word coming after (from and in). In this case, it is

country_A
country_B.

What are the tools, that I can use to do that and which are faster than others. because the log files are huge.

Thanks in advance
# 2  
Old 02-25-2011
Hi

Code:
$ cat a
Hello, I am from country_A, I am living in country_B


$ sed -e 's/.*from *\([^, ]*\).*in *\([^, ]*\).*/\1\n\2/' a
country_A
country_B

Guru
# 3  
Old 02-25-2011
Thank you very much
# 4  
Old 02-25-2011
or try yourself with a simple script Smilie

Code:
#!/bin/bash
c=0 ; ok=0
for i in `cat infile`; 
 do
  ((c++)) ;
  if [ $i = from ] || [ $i = in ] ; then let ok=$c+1 ;fi
  if [ $ok -eq $c ] ; then echo $i|sed 's/[,]$//' ; fi
 done

# 5  
Old 02-25-2011
The first code worked fine for the given example but when I tried to use it in the log files, It did not work for some reason. The files I am working on are comma separated postgres log files.

for the second code it might be a good solution but, the file sizes is about 1GB. I tried first to do the solution using grep and egrep because it is faster.

For example I changed the expression a little bit to get the table names from the log files

Code:
egrep "*[FJ][RO][OI][MN] +[^,; ]*" postgresql-xxxx.csv -o

This code is nice but I am gettin JOIN plapla or FROM plapa and I do not want to do more processing to get rid of JOIN and FROM

So I tried sed and it did not work and I get very strange result

Code:
sed -e 's_*[FJ][RO][OI][MN] +\([^,; ]*\)_\1_' postgresql-xxxx.csv


Last edited by fdc2suxs; 02-25-2011 at 02:46 PM..
# 6  
Old 02-26-2011
fdc2suxs,

Please try with this awk one:
Code:
echo "Hello, I am from country_A, I am living in country_B.
Hello, how are you? I am from country_C, but I know many people in country_D too.
Really?, and I like going from City_E to City_F, and then enjoy in City_G for a while." | awk '{
sub(".*from ","",$0);A=$1;sub(".*in ","",$0);B=$1;print A"\n"B}' | sed 's/[,.]//'
country_A
country_B
country_C
country_D
City_E
City_G

* The last sed part used to remove some extra "," or "." that could remain.

Hope it works in your big fileSmilie.
# 7  
Old 02-26-2011
Code:
# echo "Hello, I am from country_A, I am living in country_B.
Hello, how are you? I am from country_C, but I know many people in country_D too.
Really?, and I like going from City_E to City_F, and then enjoy in City_G for a while." | ruby -0777 -e 'gets.scan(/(from|in)\s+(.[^\s]*)/).each{|x| puts x[-1]}'
country_A,
country_B.
country_C,
country_D
City_E
City_G

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search a text and return the text from file

Hi I have a set of input strings in a pattern as given below string1 string2 string3 string4 string5 I need to search this sequence of strings from a file in such a way that the first two strings (string1 and string2) and last two strings (string4 and string5) should match with the... (8 Replies)
Discussion started by: my_Perl
8 Replies

2. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

3. UNIX for Dummies Questions & Answers

Search String, Out matched text and input text for no match.

I need to search a string for some specific text which is no big deal using grep. My problem is when the search fails to find the text. I need to add text like "na" when my search does not match. I have tried this command but it does not work when I put the command in a loop in a bash script: ... (12 Replies)
Discussion started by: jojojmac5
12 Replies

4. UNIX for Advanced & Expert Users

Search and replace text

HI I have property files having content QA_server_name=10.232.54.7 QA_port_number=18000 DEV_server_name=10.235.60.73 DEV_port_number=18000 and a .jason file having content like this { "server":"localhost" "port":"17000" ------ } I will get the parameter... (1 Reply)
Discussion started by: mdtausifsh
1 Replies

5. Shell Programming and Scripting

search into text

Hello, I want to search a word inside a file from a script, that is moving through a script line by line and when I find the word "hello" to cut all the text from the beginning to there and so many times successively to get the word "hello "until the end of the text. Ie if a 4 times to cut the... (6 Replies)
Discussion started by: uri_crack
6 Replies

6. Shell Programming and Scripting

Search text pattern

Hello folks, I have a text file of apache logs, that have robots.txt text with logs, i want to search all all words that are with "anyword.txt" but i dont want to search robots.txt in file. Like i have abcd.txt file X.X.X.X - - "GET... (9 Replies)
Discussion started by: learnbash
9 Replies

7. Shell Programming and Scripting

Search text from a file and print text and one previous line too

Hi, Please let me know how to find text and print text and its previous line. Please don't get irritated few days back I asked text and next line. I am using HP-UX 11.11 Thanks for your help. (6 Replies)
Discussion started by: kamranjalal
6 Replies

8. UNIX for Dummies Questions & Answers

search and replace a specific text in text file?

I have a text file with following content (3 lines) filename : output.txt first line:12/12/2008 second line:12/12/2008 third line:Y I would like to know how we can replace 'Y' with 'N' in the 3rd line keeping 1st and 2nd lines same as what it was before. I tried using cat output.txt... (4 Replies)
Discussion started by: santosham
4 Replies

9. Shell Programming and Scripting

Perl: Search for string on line then search and replace text

Hi All, I have a file that I need to be able to find a pattern match on a line, search that line for a text pattern, and replace that text. An example of 4 lines in my file is: 1. MatchText_randomNumberOfText moreData ReplaceMe moreData 2. MatchText_randomNumberOfText moreData moreData... (4 Replies)
Discussion started by: Crypto
4 Replies
Login or Register to Ask a Question