Sponsored Content
Top Forums Shell Programming and Scripting Comparing 2 files and return the unique lines in first file Post 302281105 by npatwardhan on Wednesday 28th of January 2009 07:59:38 AM
Old 01-28-2009
hi franklin, care to explain what the awk statement means? thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

comparing 2 text files to get unique values??

Hi all, I have got a problem while comparing 2 text files and the result should contains the unique values(Non repeatable). For eg: file1.txt 1 2 3 4 file2.txt 2 3 So after comaping the above 2 files I should get only 1 and 4 as the output. Pls help me out. (7 Replies)
Discussion started by: smarty86
7 Replies

2. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

3. UNIX for Dummies Questions & Answers

Getting non unique lines from concatenated files

Hi All, Is there a way to get NON unique lines from 2 or more concatenated files? Basically I have several files which are very similar with the exception of few lines and I want to find out which lines are different in each file. Very simple example is file1 contains: 1 2 3 4 5file2... (122 Replies)
Discussion started by: pawannoel
122 Replies

4. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

5. Shell Programming and Scripting

Comparing files and capture return code

Hi, I would like to compare 2 files, and have a return code write to a file. regardless of the files contents are the same the code should be writing to a file (if both files contents are same then return code 0). A simple example will be great :) Thanks (3 Replies)
Discussion started by: khchong
3 Replies

6. UNIX for Dummies Questions & Answers

getting unique lines from 2 files

hi i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command? my... (1 Reply)
Discussion started by: anurupa777
1 Replies

7. Shell Programming and Scripting

compare 2 files and return unique lines in each file (based on condition)

hi my problem is little complicated one. i have 2 files which appear like this file 1 abbsss:aa:22:34:as akl abc 1234 mkilll:as:ss:23:qs asc abc 0987 mlopii:cd:wq:24:as asd abc 7866 file2 lkoaa:as:24:32:sa alk abc 3245 lkmo:as:34:43:qs qsa abc 0987 kloia:ds:45:56:sa acq abc 7805 i... (5 Replies)
Discussion started by: anurupa777
5 Replies

8. Shell Programming and Scripting

Comparing 2 text files & downloading a file if the last lines are different

Hello I'm having a little difficulty in writing a shell script for a few simple tasks. First I have two files "file1.txt" and "file2.txt" and I want to read and compare the last line of each file. The files look like this. File1.txt File2.txt After comparing the two lines I would... (2 Replies)
Discussion started by: RustikGaming
2 Replies

9. Shell Programming and Scripting

Using grep and a parameter file to return unique values

Hello Everyone! I have updated the first post so that my intentions are easier to understand, and also attached sample files (post #18). I have over 500 text files in a directory. Over 1 GB of data. The data in those files is organised in lines: My intention is to return one line per... (23 Replies)
Discussion started by: clippertm
23 Replies

10. Shell Programming and Scripting

Shell Scripting | Return list of unique characters in files

Hi, I am trying to script the below, but I am not very good at it :( Your help would be greatly appreciated. 1. read all files in the directory in strings strings *.* 2. in each file, for each line that contains "ABCD", store characters located at position 521 and 522 of this line... (9 Replies)
Discussion started by: clippertm
9 Replies
EXIMSTATS(8)							       EXIM4							      EXIMSTATS(8)

NAME
eximstats - generates statistics from Exim mainlog or syslog files. SYNOPSIS
eximstats [Output] [Options] mainlog1 mainlog2 ... eximstats -merge [Options] report.1.txt report.2.txt ... > weekly_report.txt Output: -txt Output the results in plain text to STDOUT. -txt=filename Output the results in plain text. Filename '-' for STDOUT is accepted. -html Output the results in HTML to STDOUT. -html=filename Output the results in HTML. Filename '-' for STDOUT is accepted. -xls Output the results in Excel compatible Format to STDOUT. Requires the Spreadsheet::WriteExcel CPAN module. -xls=filename Output the results in Excel compatible format. Filename '-' for STDOUT is accepted. Options: -hnumber histogram divisions per hour. The default is 1, and 0 suppresses histograms. Valid values are: 0, 1, 2, 3, 5, 10, 15, 20, 30 or 60. -ne Don't display error information. -nr Don't display relaying information. -nr/pattern/ Don't display relaying information that matches. -nt Don't display transport information. -nt/pattern/ Don't display transport information that matches -qlist List of times for queuing information single 0 item suppresses. -tnumber Display top <number> sources/destinations default is 50, 0 suppresses top listing. -tnl Omit local sources/destinations in top listing. -t_remote_users Include remote users in the top source/destination listings. -include_original_destination Include the original destination email addresses rather than just using the final ones. Useful for finding out which of your mailing lists are receiving mail. -show_dtlist Show the delivery times (DT)for all the messages. Exim must have been configured to use the +deliver_time logging option for this option to work. list is an optional list of times. Eg -show_dt1,2,4,8 will show the number of messages with delivery times under 1 second, 2 seconds, 4 seconds, 8 seconds, and over 8 seconds. -show_rtlist Show the receipt times for all the messages. The receipt time is defined as the Completed hh:mm:ss - queue_time_overall - the Receipt hh:mm:ss. These figures will be skewed by pipelined messages so might not be that useful. Exim must have been configured to use the +queue_time_overall logging option for this option to work. list is an optional list of times. Eg -show_rt1,2,4,8 will show the number of messages with receipt times under 1 second, 2 seconds, 4 seconds, 8 seconds, and over 8 seconds. -byhost Show results by sending host. This may be combined with -bydomain and/or -byemail and/or -byedomain. If none of these options are specified, then -byhost is assumed as a default. -bydomain Show results by sending domain. May be combined with -byhost and/or -byemail and/or -byedomain. -byemail Show results by sender's email address. May be combined with -byhost and/or -bydomain and/or -byedomain. -byemaildomain or -byedomain Show results by sender's email domain. May be combined with -byhost and/or -bydomain and/or -byemail. -pattern Description /Pattern/ Look for the specified pattern and count the number of lines in which it appears. This option can be specified multiple times. Eg: -pattern 'Refused connections' '/refused connection/' -merge This option allows eximstats to merge old eximstat reports together. Eg: eximstats mainlog.sun > report.sun.txt eximstats mainlog.mon > report.mon.txt eximstats mainlog.tue > report.tue.txt eximstats mainlog.wed > report.web.txt eximstats mainlog.thu > report.thu.txt eximstats mainlog.fri > report.fri.txt eximstats mainlog.sat > report.sat.txt eximstats -merge report.*.txt > weekly_report.txt eximstats -merge -html report.*.txt > weekly_report.html o You can merge text or html reports and output the results as text or html. o You can use all the normal eximstat output options, but only data included in the original reports can be shown! o When merging reports, some loss of accuracy may occur in the top n lists. This will be towards the ends of the lists. o The order of items in the top n lists may vary when the data volumes round to the same value. -charts Create graphical charts to be displayed in HTML output. Only valid in combination with -html. This requires the following modules which can be obtained from http://www.cpan.org/modules/01modules.index.html GD GDTextUtil GDGraph To install these, download and unpack them, then use the normal perl installation procedure: perl Makefile.PL make make test make install On Debian GNU/Linux you can use "apt-get install libgd-perl libgd-text-perl libgd-graph-perl" instead. -chartdirI <dir> Create the charts in the directory <dir> -chartrelI <dir> Specify the relative directory for the "img src=" tags from where to include the charts -emptyok Specify that it's OK to not find any valid log lines. Without this we will output an error message if we don't find any. -d Debug flag. This outputs the eval()'d parser onto STDOUT which makes it easier to trap errors in the eval section. Remember to add 1 to the line numbers to allow for the title! DESCRIPTION
Eximstats parses exim mainlog and syslog files to output a statistical analysis of the messages processed. By default, a text analysis is generated, but you can request other output formats using flags. See the help (-help) to learn about how to create charts from the tables. AUTHOR
There is a web site at http://www.exim.org TO DO
This program does not perfectly handle messages whose received and delivered log lines are in different files, which can happen when you have multiple mail servers and a message cannot be immediately delivered. Fixing this could be tricky... Merging of xls files is not (yet) possible. Be free to implement :) perl v5.14.2 2014-07-24 EXIMSTATS(8)
All times are GMT -4. The time now is 04:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy