Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to grep a log file for words listed in separate text file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to grep a log file for words listed in separate text file?
# 1  
Old 09-11-2013
[SOLVED] How to grep a log file for words listed in separate text file?
Hello,

I want to grep a log ("server.log") for words in a separate file ("white-list.txt") and generate a separate log file containing each line that uses a word from the "white-list.txt" file.

Putting that in bullet points:
  • Search through "server.log" for lines that contain any word contained within "white-list.txt" file
  • Create separate file ("player_activity.log")
I've done this before with this command line:
Code:
grep -E "player1|player2|player3" server.log >player_activity.log

But I'm tired of modifying that line and would just like to draw from my known player list in a different file. I want it to substitute the "player1|player2|player3" with looking through a white-list.txt that looks like:

Code:
player1
player2
player3
player4
player5
player6

The output should look like this:
Code:
2013-08-11 08:46:23 [INFO] player1[/xxx.xxx.xxx.xxx:53531] logged in with entity id 1072 at ([regolare] -220.7308956951062, 62.0, -108.79669524885138)
2013-08-11 08:46:23 [INFO] Player joined: player1
2013-08-11 08:50:50 [INFO] player2[/xxx.xxx.xxx.xxx:53019] logged in with entity id 5685 at ([mundusprime] 1995.377727365253, 75.0, 883.1106329044326)
2013-08-11 08:50:50 [INFO] Player joined: player2
2013-08-11 08:51:00 [INFO] <player1> morning
2013-08-11 08:51:06 [INFO] <player2> hola mi compadre!
2013-08-11 08:51:31 [INFO] <player2> how goes the pyramid?
2013-08-11 08:51:53 [INFO] <player1> roof is complete, need to finish clearing interior.
2013-08-11 08:52:06 [INFO] <player2> I took a peek the other day, it's impressive!

Last edited by nbsparks; 09-11-2013 at 08:11 PM..
# 2  
Old 09-11-2013
Code:
grep -f white-list.txt server.log

white-list.txt:
Code:
player1
player2
player3
player4
player5
player6

This User Gave Thanks to Scott For This Post:
nbsparks
# 3  
Old 09-11-2013
[SOLVED] How to grep a log file for words listed in separate text file?
OK... That was way too simple. Thanks.
Last edited by nbsparks; 09-11-2013 at 08:31 PM..
# 4  
Old 09-17-2013
OK... Now I have a new spin-off issue. Is there a way to grep "file a" (a log file) for names held in "file b" (a list of usernames) AND also file "c" (a list of specific phrases of interest that could show up in the log - i.e. showing login attempts from non approved users)?

I would do it as two separate grep statements, but I would like the results to be mixed together in chronological order from the log file.
# 5  
Old 09-17-2013
Code:
$ grep -f names <(grep -f phrases mainfile)

# 6  
Old 09-17-2013
Quote:
Originally Posted by Scott
Code:
$ grep -f names <(grep -f phrases mainfile)

So, "names" would be my "white-list.txt" full of usernames?
And "phrases" would be my list of phrases of concern?
And "mainfile" would be the server.log that I'm parsing?
# 7  
Old 09-17-2013
That's the theory Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Want to delete the junk files from a directory which are not listed in a TEXT file

Hello Everyone, I want to delete the image files from a directory, which are not listed in a TEXT file. The directory contains large number of image files (in millions) required / not required. I want to delete the image files which are "not required". I have generated a Text file having... (3 Replies)
Discussion started by: Praveen Pandit
3 Replies

2. Shell Programming and Scripting

Cat files listed in text file and redirect to new directory with same filename

I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies

3. Shell Programming and Scripting

Delete files listed in text file

Hi Team, Here's the scenario, I have a text file called "file_list.txt". Its content is as follows. 111.tmp 112.tmp 113.tmp 114.tmp These files will present in "workdir" directory. It has many files. But only the files present in file_list.txt has to be deleted from the workdir... (7 Replies)
Discussion started by: kmanivan82
7 Replies

4. Shell Programming and Scripting

Copy files listed in text file to new directory

I am trying to write a script that will copy all file listed in a text file (100s of file names) to a new directory Assume script will run with main as current working directory and I know how many files/lines will be in List.txt Im trying to work up a test script using this model Contents of... (2 Replies)
Discussion started by: IAmTheGrass
2 Replies

5. Shell Programming and Scripting

Send a mail to IDs listed in a text file

I have a list of mail ids in text file and want a ksh script that reads this text file and sends a mail to all mail ids with same subject line and content. I am using UX-HP machine and KSH. Thanks for help in advance! (5 Replies)
Discussion started by: Sriranga
5 Replies

6. Shell Programming and Scripting

Shellscript to sort duplicate files listed in a text file

I have many pdf's scattered across 4 machines. There is 1 location where I have other Pdf's maintained. But the issues it the 4 machines may have duplicate pdf's among themselves, but I want just 1 copy of each so that they can be transfered to that 1 location. What I have thought is: 1) I have... (11 Replies)
Discussion started by: deaddevil
11 Replies

7. Shell Programming and Scripting

Copy files listed in a text file - whitespace problem.

Hi, Say I have this text file <copy.out> that contains a list of files/directories to be copied out to a different location. $ more copy.out dir1/file1 dir1/file2 dir1/file3 "dir1/white space" dir1/file4 If I do the following: $copy=`more copy.out` $echo $copy dir1/file1... (4 Replies)
Discussion started by: 60doses
4 Replies

8. UNIX for Dummies Questions & Answers

how to separate numbers and words from a file using shell scripts

Hi, How to separate numbers and words(with full alphabets) in a particular file and store it in two different files. Please help me out for this.Using shell scripting. :confused::confused: (1 Reply)
Discussion started by: kamakshi s
1 Replies

9. Shell Programming and Scripting

To read and separate number and words in file and store to two new file using shell

hi, I am a begginer in unix and i want to know how to open a file and read it and separate the numbers & words and storing it in separate files, Using shell scripting. Please help me out for this. Regards S.Kamakshi (2 Replies)
Discussion started by: kamakshi s
2 Replies

10. UNIX for Dummies Questions & Answers

Listing words from a file on a Separate Line

Hi, I want to list all the words in my file on a separate line. I am using the bourne(sh)/bourne again shell(bash). Thanks, theA (2 Replies)
Discussion started by: Astudent
2 Replies
Login or Register to Ask a Question