Sponsored Content
Top Forums Shell Programming and Scripting Joining three files with uniq no. Post 302735395 by binlib on Saturday 24th of November 2012 05:19:05 PM
Old 11-24-2012
With input files so big, it's more efficient to use join:
Code:
join -t\| -j1 -o1.1,1.2,1.3,1.4,1.5,1.6,1.7,2.2,2.3,2.4 f1 f2 |join -t\| -j1 - f3

If the input files were not sorted, sort first before joining. The long list of fields after -o is to get rid of the trailing '|' from f1, otherwise, the whole -o... can be omitted.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

joining 2 files

Hi, I have two files that I need to find difference between. Do I use diff or join? If join, how do I use it? thanks, webtekie (1 Reply)
Discussion started by: webtekie
1 Replies

2. Shell Programming and Scripting

Help with joining two files

Greetings, all. I've got a project that requires I join two data files together, then do some processing and output. Everything must be done in a shell script, using standard unix tools. The files look like the following: File_1 Layout: Acct#,Subacct#,Descrip Sample: ... (3 Replies)
Discussion started by: rjlohman
3 Replies

3. Shell Programming and Scripting

Joining files

Hi, Whats the unix function to join multiple files? is it cat? so I have multiple files in the same format and I want to join then by row eg. FILE1 1 3 1 3 1 3 1 3 FILE2 2 4 2 4 2 4 (1 Reply)
Discussion started by: kylle345
1 Replies

4. Shell Programming and Scripting

Using uniq on log files

I have this log file which I need to count the number of repeated line and do some manipulation. test.log: June 3 03:33:38 test 1 June 3 10:31:22 test 2 June 3 10:32:22 test 2 June 3 10:33:22 test 3 June 3 10:33:22 test 3 June 3 10:34:22 test 4 June 3 10:35:22 test 5 ... (4 Replies)
Discussion started by: jazzaddict
4 Replies

5. Shell Programming and Scripting

joining two or more files

i have three files file a has contents 123 234 238 file b has contents 189 567 567 and file c has contents qwe ert ery (1 Reply)
Discussion started by: tomjones
1 Replies

6. Shell Programming and Scripting

Joining Three Files

Hi guys, I have three files which needs to be joined to a single file. File 1: Col a, Col b, Col c File 2: Col 1a, Col 1b File 3: Col 2a, Col 2b Output: Col 1a, Col 2a, Col a, Col b, Col c. All the files are comma delimited. I need to join Col b with Col 1b and need to... (17 Replies)
Discussion started by: mac4rfree
17 Replies

7. Shell Programming and Scripting

Joining two files into one

Hi experts, I'm quite newbie here!! I have two seperate files. Contents of file like below File 1: 6213019212001 8063737 File:2 15703784 I want to join these two files into one where content will be File 3: 6213019212001 8063737 15703784 Regards, Ray Seilden (1 Reply)
Discussion started by: RayanS
1 Replies

8. Shell Programming and Scripting

Help with joining files and adding headers to files

Hi, I have about 20 tab delimited text files that have non sequential numbering such as: UCD2.summary.txt UCD45.summary.txt UCD56.summery.txt The first column of each file has the same number of lines and content. The next 2 column have data points: i.e UCD2.summary.txt: a 8.9 ... (8 Replies)
Discussion started by: rrdavis
8 Replies

9. Shell Programming and Scripting

Joining 2 Files

File "A" (column names: Nickname Number GB) Nickname Number GB PROD_DB0034 100A 16 ASMIL1B_DATA_003 100B 16 PSPROD_0000 1014 36 PSPROD_0001 100D 223 ..... File "B" (column names: TYPE DEVICE NUMBER SIZE) TYPE DEVICE NUMBER SIZE 1750500 hdisk2 100A 16384 1750500 hdisk3 ... (4 Replies)
Discussion started by: Daniel Gate
4 Replies

10. Shell Programming and Scripting

Please help me in joining two files

I have two files with the below contents : sampleoutput3.txt 20150202;hostname1 20150223;hostname2 20150716;hostname3 sampleoutput1.txt hostname;packages_out_of_date;errata_out_of_date; hostname1;11;0; hostnamea;12;0; hostnameb;11;0; hostnamec;95;38; hostnamed;440;358;... (2 Replies)
Discussion started by: rahul2662
2 Replies
JOIN(1) 							   User Commands							   JOIN(1)

NAME
join - join lines of two files on a common field SYNOPSIS
join [OPTION]... FILE1 FILE2 DESCRIPTION
For each pair of input lines with identical join fields, write a line to standard output. The default join field is the first, delimited by whitespace. When FILE1 or FILE2 (not both) is -, read standard input. -a FILENUM also print unpairable lines from file FILENUM, where FILENUM is 1 or 2, corresponding to FILE1 or FILE2 -e EMPTY replace missing input fields with EMPTY -i, --ignore-case ignore differences in case when comparing fields -j FIELD equivalent to '-1 FIELD -2 FIELD' -o FORMAT obey FORMAT while constructing output line -t CHAR use CHAR as input and output field separator -v FILENUM like -a FILENUM, but suppress joined output lines -1 FIELD join on this FIELD of file 1 -2 FIELD join on this FIELD of file 2 --check-order check that the input is correctly sorted, even if all input lines are pairable --nocheck-order do not check that the input is correctly sorted --header treat the first line in each file as field headers, print them without trying to pair them -z, --zero-terminated end lines with 0 byte, not newline --help display this help and exit --version output version information and exit Unless -t CHAR is given, leading blanks separate fields and are ignored, else fields are separated by CHAR. Any FIELD is a field number counted from 1. FORMAT is one or more comma or blank separated specifications, each being 'FILENUM.FIELD' or '0'. Default FORMAT outputs the join field, the remaining fields from FILE1, the remaining fields from FILE2, all separated by CHAR. If FORMAT is the keyword 'auto', then the first line of each file determines the number of fields output for each line. Important: FILE1 and FILE2 must be sorted on the join fields. E.g., use "sort -k 1b,1" if 'join' has no options, or use "join -t ''" if 'sort' has no options. Note, comparisons honor the rules specified by 'LC_COLLATE'. If the input is not sorted and some lines cannot be joined, a warning message will be given. GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report join translation bugs to <http://translationproject.org/team/> AUTHOR
Written by Mike Haertel. COPYRIGHT
Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
comm(1), uniq(1) The full documentation for join is maintained as a Texinfo manual. If the info and join programs are properly installed at your site, the command info coreutils 'join invocation' should give you access to the complete manual. GNU coreutils 8.22 June 2014 JOIN(1)
All times are GMT -4. The time now is 11:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy