Using your sample data files, here is an example of the use of join, which is unlikely to run into memory limits. There is a lot of set-up and comparison code, so just look at the heart of the computation, the join:
producing:
The join utility requires files ordered on the join field. This means that the result might be out of order with respect to the original positions of lines in the file. If so, you will need to re-order, say on the line number field.
Hi guys,
firstly I'm working on SunOS 5.10 Generic_125100-10 sun4u sparc SUNW,Sun-Fire-V240
I've made a script to compress two directory and then send them to an other server via ftp. This is working very well.
Inside theis script I decide to log usefull data for troubleshooting in case of... (7 Replies)
I am basically a scripting noob, I have some programming logic, and I wouldn't post here if my 3 hours of searching actually found something.
So far this is what I have:
"
#! /bin/ksh
List=./pinglist1.txt
cat $List | while read ip
do
Pingable=""
ping $ip -n 2 | awk '/100%/ {print... (11 Replies)
Hi every body
i have a problem need help urgently
file 1 (approx 200K entries)
aaaaa
bbbb
cccccc
dddd
ffff
file 2 (approx 2 million entries)
aaaaa,1,ee,44,5t,6y,
bbbb,3,ff,66,5u,8r,
cccccc, .....
dddd, .....
eeeeee, .....
ffff, ...... (5 Replies)
Hi,
I have two files , one file with data file with attributes that need to be sent to another file to generate a predefined format.
Example:
File.txt
AP|{SSHA}VEEg42CNCghUnGhCVg==
APVG3|{SSHA}XK|"password"
AP3|{SSHA}XK|"This is test"
....
etc
---------
test.sh has... (1 Reply)
Hi Friends,
I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving...
File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Hi,
I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file.
File1 - file2 = file3
wc -l file1.txt
58112
wc -l file2.txt
55260
head -5 file1.txt
101214200123
101214700300
101250030067
101214100500... (10 Replies)
Hi
Please help me to compare two files and output into a new file
file1.txt
15114933 |4001
15291649 |933502
15764675 |4316
15764678 |4316
15761974 |282501
15673104 |933505
15673577 |933505
15673098 |933505
15673096 |933505
15673092 |933505
15760705 ... (13 Replies)
I am working on an outage script and I run a command from the command line which tells me the amount of generator failures in my market. The output of this command only gives me three digits to identify the site by. I have a master list of all sites in a separate file, call it list.txt. If my... (7 Replies)
Hi guys,
I have a text file named file1.txt that is formatted like this:
001 , ID , 20000
002 , Name , Brandon
003 , Phone_Number , 616-234-1999
004 , SSNumber , 234-23-234
005 , Model , Toyota
007 , Engine ,V8
008 , GPS , OFF
and I have file2.txt formatted like this:
... (2 Replies)
Shell script logic
Hi
I have 2 input files like with file 1 content as (file1)
"BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt
File 2 contents as fle(2)
"BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt
I want to get... (22 Replies)
Discussion started by: pottic
22 Replies
LEARN ABOUT X11R4
comm
COMM(1) User Commands COMM(1)NAME
comm - compare two sorted files line by line
SYNOPSIS
comm [OPTION]... FILE1 FILE2
DESCRIPTION
Compare sorted files FILE1 and FILE2 line by line.
When FILE1 or FILE2 (not both) is -, read standard input.
With no options, produce three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and
column three contains lines common to both files.
-1 suppress column 1 (lines unique to FILE1)
-2 suppress column 2 (lines unique to FILE2)
-3 suppress column 3 (lines that appear in both files)
--check-order
check that the input is correctly sorted, even if all input lines are pairable
--nocheck-order
do not check that the input is correctly sorted
--output-delimiter=STR
separate columns with STR
--total
output a summary
-z, --zero-terminated
line delimiter is NUL, not newline
--help display this help and exit
--version
output version information and exit
Note, comparisons honor the rules specified by 'LC_COLLATE'.
EXAMPLES
comm -12 file1 file2
Print only lines present in both file1 and file2.
comm -3 file1 file2
Print lines in file1 not in file2, and vice versa.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report comm translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO join(1), uniq(1)
Full documentation at: <http://www.gnu.org/software/coreutils/comm>
or available locally via: info '(coreutils) comm invocation'
GNU coreutils 8.28 January 2018 COMM(1)