Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Unique entries in multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unique entries in multiple files
# 15  
Old 09-14-2015
Yes, I overlooked something. Correction:
Code:
sed -n "s/^.*from=<\([^@]*\)@example.com.*/\1/p" file

# 16  
Old 09-14-2015
Unfortunatelly it is still not working.
Code:
#!/bin/bash
DIR=/usr/src/count_test3
FILES=$DIR/*
for i in $FILES
    do

        sed -n "s/^.*from=<\([^@]*\)@example.com.*/\1/p" $i | sort | uniq -c
                   done

I've got nothing.
# 17  
Old 09-14-2015
Does it work on the sample in post#12?
# 18  
Old 09-14-2015
Yes, but it doesn't work on other log file.
It can look like this:

Code:
Sep 11 12:51:18 servername postfix/qmgr[18292]: 61F981C3A39: from=<ramirez987@example.com>, size=490403, nrcpt=1 (queue active)
Sep 11 12:51:18 servername mimedefang.pl[25646]: times int: 0,0,0,0,0,0 0
Sep 11 12:51:18 servername mimedefang.pl[25646]: checking by antivirus
Sep 11 12:51:18 servername postfix/smtpd[26146]: D3A771C39AA: client=emsdgdsilgw1sdg.passgel.com[xxx.xxx.xxx.xxx]
Sep 11 12:51:18 servername postfix/smtpd[26179]: disconnect from maidgssdggewwle.com[xxx.xxx.xxx.xxx]
Sep 11 12:51:18 servername postfix/virtual[23973]: 61F981C3A39: to=<john@nothing.com>, relay=virtual, delay=2.5, delays=2.4/0/0/0.14, dsn=2.0.0, status=sent (delivered to mai$
Sep 11 12:51:18 servername postfix/qmgr[18292]: 61F981C3A39: removed
Sep 11 12:51:18 servername mimedefang.pl[25646]: times: 0,0,0,0,0
Sep 11 12:51:18 servername mimedefang.pl[25646]: MDLOG,0495B1C3A2D,mail_in,,,<sdfr@newssfer.furn.com>,<lucas@example.com>,=?utf-8?q?Sprawd=C5=BA_aktualne_M$
Sep 11 12:51:18 servername mimedefang.pl[25646]: autoresponse1  sdfr@newssfer.furn.com-> lucas@example.com sdfr@newssfer.furn.com
Sep 11 12:51:18 servername mimedefang.pl[25646]: FILTER_END sender <sdfr@newssfer.furn.com> relayAddr xxx.xxx.xxx.xxx relayHostN: [xxx.xxx.xxx.xxx] helo: mail3.m3.pl rcpt$
Sep 11 12:51:19 servername postfix/qmgr[18292]: 0495B1C3A2D: from=<sdfr@newssfer.furn.com>, size=445850, nrcpt=1 (queue active)
Sep 11 12:51:21 servername postfix/qmgr[18292]: D3A771C39AA: from=<james@example.com>, size=46390, nrcpt=3 (queue active)
Sep 11 12:51:21 servername postfix/virtual[23973]: D3A771C39AA: to=<blablablah@nothing.com>, relay=virtual, delay=3.4, delays=3.4/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to m$
Sep 11 12:51:21 servername postfix/virtual[23973]: D3A771C39AA: to=<abc@nothing.com>, relay=virtual, delay=3.4, delays=3.4/0.01/0/0.02, dsn=2.0.0, status=sent (delive$
Sep 11 12:51:21 servername postfix/virtual[23973]: D3A771C39AA: to=<def@nothing.com>, relay=virtual, delay=3.4, delays=3.4/0.01/0/0.02, dsn=2.0.0, status=sent (deliver$
Sep 11 12:51:21 servername postfix/qmgr[18292]: D3A771C39AA: removed

Last edited by ramirez987; 09-14-2015 at 06:38 AM..
# 19  
Old 09-14-2015
Works perfectly for me:
Code:
sed -n "s/^.*from=<\([^@]*\)@example.com.*/\1/p" file
ramirez987
james

WHAT (and HOW) "doesn't work"?
# 20  
Old 09-14-2015
Quote:
Originally Posted by RudiC
Works perfectly for me:
Code:
sed -n "s/^.*from=<\([^@]*\)@example.com.*/\1/p" file
ramirez987
james

WHAT (and HOW) "doesn't work"?
Well... I have no idea. It is working perfectly fine with the few lines above, but when I try to analyze full log file, there is no result.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split into multiple files by using Unique columns in a UNIX file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|1|1 123|12|5679|BCDEFG|34CD56|2019-07-10|2019-07-10|896.7200|1|2... (3 Replies)
Discussion started by: RVSP
3 Replies

2. Shell Programming and Scripting

Getting unique sequences from multiple fasta file

Hi, I have a fasta file with multiple sequences. How can i get only unique sequences from the file. For example my_file.fasta >seq1 TCTCAAAGAAAGCTGTGCTGCATACTGTACAAAACTTTGTCTGGAGAGATGGAGAATCTCATTGACTTTACAGGTGTGGACGGTCTTCAGAGATGGCTCAAGCTAACATTCCCTGACACACCTATAGGGAAAGAGCTAAC >seq2... (3 Replies)
Discussion started by: Ibk
3 Replies

3. Shell Programming and Scripting

Sort unique by multiple fields

i need to sort to get all the unique records based on the 1st and 2nd column, and keep the record with the highest value on 5th column if there are duplicates, every column with varies length a^2^x^y^z bxc^2xx2^aa^bvxxxx^cdd a^3^1^2^3 a^2^x^1^c I want a result which will only keep the 1st... (2 Replies)
Discussion started by: dtdt
2 Replies

4. Shell Programming and Scripting

Count Unique values from multiple lists of files

Looking for a little help here. I have 1000's of text files within a multiple folders. YYYY/ /MM /1000's Files Eg. 2014/01/1000 files 2014/02/1237 files 2014/03/1400 files There are folders for each year and each month, and within each monthly folder there are... (4 Replies)
Discussion started by: whegra
4 Replies

5. Shell Programming and Scripting

Reading multiple values from multiple lines and columns and setting them to unique variables.

Hello, I would like to ask for help with csh script. An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Discussion started by: FMMOLA
7 Replies

6. Shell Programming and Scripting

Compare columns of multiple files and print those unique string from File1 in an output file.

Hi, I have multiple files that each contain one column of strings: File1: 123abc 456def 789ghi File2: 123abc 456def 891jkl File3: 234mno 123abc 456def In total I have 25 of these type of file. (5 Replies)
Discussion started by: owwow14
5 Replies

7. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

8. Shell Programming and Scripting

Deriving unique entries from multiple repeating patterns

Dear All, I have a below one column data.(example) Col1 1 2 . . 25 8 9 25 1 2 . . 25 Where each entry(row) is a number from 1-25, but in place whereever mentioned with . we have all the entries 1-25, but some places where ever no . like in 8 9 25 I have only 3 entries. No I... (14 Replies)
Discussion started by: ks_reddy
14 Replies

9. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

10. Shell Programming and Scripting

adding a column of unique string to multiple files

Hi, I need to add a column of unique string to individual csv file. my individual csv file looks like this and each file has variable row length: 178.52,39.4,train,0.003355544375334351,39.15752753933254,0.4895933968953914... (7 Replies)
Discussion started by: ida1215
7 Replies
Login or Register to Ask a Question