Complex grep command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Complex grep command
# 1  
Old 11-18-2014
Complex grep command

Hallo Team,

I need your help and its rather urgent.
I have a file with thousands of lines. Here is a sample below:

Code:
Sample1.txt

BW235045560121114833444044@196.35.130.5
BW235106757121114-574455394@196.35.130.5
BW2349514941211141077771352@196.35.130.5
BW2353526471211141758512647@196.35.130.5
BW235328223121114-1191893748@196.35.130.5
BW235101704121114-533816579@196.35.130.5
BW235258139121114-1086573027@196.35.130.5
BW235030487121114-1509234107@196.35.130.5
BW234902738121114-471354714@196.35.130.5
BW235128727121114-1903901078@196.35.130.5
BW2349569391211141739226705@196.35.130.5
BW235412196121114-480323232@196.35.130.5
BW2350437861211141154590213@196.35.130.5

I need to grep for each pattern on the file for example:
I took the first pattern on sample.txt to make an example of what i would like to achieve.
Code:
grep "BW235045560121114833444044@196.35.130.5" BW*20141112*.csv >> /home/paxk1/test.csv

I hope this is clear. Let me know if you need more explanation.

Regards and thanking you in advance,

Pax
# 2  
Old 11-18-2014
Quite a basic task, isnt it?

(untested)
Code:
SRC=Sample1.txt
PARSE=BW*20141112*.csv
OUT=/home/paxk1/test.csv

while read pattern;do
     grep "$pattern" $PARSE >> "$OUT"
done<"$SRC"

Hope this helps
This User Gave Thanks to sea For This Post:
# 3  
Old 11-18-2014
Sea, i guess i will have to initialize $pattern right?

so should the code be like :
Code:
SRC=Sample1.txt
PARSE=BW*20141112*.csv
OUT=/home/paxk1/test.csv
PATTERN=BW

while read pattern;do
     grep "$PATTERN" $PARSE >> "$OUT"
done<"$SRC"

?
# 4  
Old 11-18-2014
No, pattern is being read from $SRC which points to your sample1.txt . What's the result of running sea's code as is?
This User Gave Thanks to RudiC For This Post:
# 5  
Old 11-18-2014
If one reads your requirement:
Quote:
I need to grep for each pattern on the file for example:
Would it be fair to say that you want to split the original files into N new files, one file for each unique line (pattern) of the original? If so, you do realize that each file will contain one or more copies of the same line (pattern). Perhaps:
Code:
sort file | uniq -c

might be useful? However, if you have to split the files:
Code:
sort -u ${FILE} | while read pattern; do
  grep "^${pattern}$" ${FILE} > "${pattern}.csv"
done

(untested)

which seems clunky since the file is being read N times, one for each pattern. How about:
Code:
sort ${FILE} | uniq -c | while read N pattern; do
  while [[ 0 -lt ${N} ]]; do
    echo "${pattern}"
    (( N = N - 1 ))
  done > "${pattern}.csv"
done

(untested)
# 6  
Old 11-20-2014
Kekanap, Have you soved it?
what i didn't got it, is where are you searching for the paterns ? on another file or on several other files ? grep output will be different !

if you are searching for exact same lines in two different files (as the pattern you get from the full line from file1) it might be much easier to just merge the 2 files and look for duplicated lines. that's 1 liner code.

Code:
cat file1.txt file2.txt | sort | uniq -c | grep " 2 "

please let us know if you solved it!

BTW one last comment. I'm not very confident to use grep with patterns that will have "." in it.

Last edited by Canu; 11-20-2014 at 04:41 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Complex Filter using grep, awk or sed

Hi, I'm not very familiar witrh sed or awk and hope the somebody can help me to solve my problem. I need to filter a text report using grep, sed or awk. I would like to cut out text lines with the pattern INFO and if exists the following lines of the pattern DETAILS. I need te keep the lines with... (4 Replies)
Discussion started by: Frankg
4 Replies

2. Shell Programming and Scripting

How to grep for a complex String?

Hi, I have a file hello.log which has the below entry ./logs/mymac/myserver.log:####<Jun 7, 2015 12:56:54 PM EDT> <myserver.my.bank.com> <mymac> < ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <1434640> <BEA-0900> <User user1 in security realm... (3 Replies)
Discussion started by: shifahim
3 Replies

3. Shell Programming and Scripting

find command with complex logic

I'm looking to write a script that will do a find of directories and delete them if they are older than x days but keep the last x # of folders even if they are older than x days. The usage is for a deployment location, so we want to keep the location clean but retain maybe the last 2 builds that... (5 Replies)
Discussion started by: MaureenT
5 Replies

4. UNIX for Advanced & Expert Users

complex find command

Hi all, I am trying to execute the following command: find 'path' -ls -exec cksum {} \; As you can see this simply finds files from a given path and runs cksum on them. My problem is this, if i have a FIFO in a directory the find tries to execute cksum on it and gets stuck. From the man page i... (9 Replies)
Discussion started by: noam128
9 Replies

5. Shell Programming and Scripting

Complex Email command in unix

Folks, I have two files, i.e. one.txt and two.dat on unix file system. I need to include the contents of one.txt in the body of the email to the sender and attach two.dat in the same email as an attachment using ksh commands, mailx or uuencode etc... I know how to send them seperately but I... (2 Replies)
Discussion started by: calredd
2 Replies

6. Shell Programming and Scripting

Complex find grep or sed command

Haven't worked in bash for ages. did a good bit of shell scripting in regular sh, but have forgotten most of it. I have several thousand php files that now include the following line at the end of the file. There is no LF or CR/LF before it begins, it is just concatenated to the final line of... (3 Replies)
Discussion started by: sjburden
3 Replies

7. Shell Programming and Scripting

complex grep command

hi all i have file call "list.log" which contains like this 00300 000024501043846 0 00300 000034531322871 0 00600 000000489100734 0 and so on .. the file goes like this:(example first row) from position 1-5 the lider number(300),position 7-21 id... (0 Replies)
Discussion started by: naamas03
0 Replies

8. Shell Programming and Scripting

complex command substitution

hi, I have to execute this line below from within a shell script; simply backquoting it is not doing the trick; it is mangling up all the options; but when i type it out on a command line, it executes cleanly. Please help me in getting this right; $ vlc -I dummy --sout='#transcode{vcodec=mp4v,... (5 Replies)
Discussion started by: spopuri
5 Replies

9. Answers to Frequently Asked Questions

advanced/complex uses of the find command

Perhaps the number one advanced find question is: How to stop find from descending into subdirectories? find command Performing a non-recursive find in Unix Use -prune with find command on AIX Searching for files over 30 days old in current directory disk space used for files with in a... (0 Replies)
Discussion started by: Perderabo
0 Replies

10. UNIX for Dummies Questions & Answers

parse text or complex grep ?

I have the following file (records from db) and I need to retrieve in another file only two variables and their values. I have a command but works only for one value. grep ^mob: R25-subs.ldi | sed 's/mob: //' | sort | uniq >text_output Can someone please help me? dn:... (4 Replies)
Discussion started by: jacost
4 Replies
Login or Register to Ask a Question