Sponsored Content
Top Forums Shell Programming and Scripting simple join for multiple files and produce 3 outputs Post 302449064 by Franklin52 on Saturday 28th of August 2010 07:21:56 AM
Old 08-28-2010
Try this, the variable nfiles contains the number of files:
Code:
awk -v nfiles="4" 'NR==FNR{a[$0]++;next}
$0 in a {a[$0]++; next}
{b[$0]++}
END{
  for(i in a){
    if(a[i]==nfiles) {
      print i > "output1"
    }
    else if(a[i]==1) {
        print i > "output3"
    }
  }
  for(i in b){
    if(b[i]==nfiles-1) {
        print i > "output2"
    }
  }
}' file1 filea fileb filec

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7... (3 Replies)
Discussion started by: InfoSeeker
3 Replies

2. Shell Programming and Scripting

How to join multiple files?

I am trying to join a few hundred files using join. Is there a way to use while read or something else to automate this. My problem is the following. Day 1 City Temp ABC 20 DEF 30 HIJ 15 Day 2 City Temp ABC 22 DEF 29 KLM 5 Day 3 (3 Replies)
Discussion started by: theFinn
3 Replies

3. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace... (19 Replies)
Discussion started by: rich@ardz
19 Replies

4. Shell Programming and Scripting

Shell script that will compare two config files and produce 2 outputs 1)actual config file 2)report

Hi I am new to shell scripting. There is a requirement to write a shell script to meet follwing needs.Prompt reply shall be highly appreciated. script that will compare two config files and produce 2 outputs - actual config file and a report indicating changes made. OS :Susi linux ver 10.3. ... (4 Replies)
Discussion started by: muraliinfy04
4 Replies

5. Shell Programming and Scripting

Awk - join multiple files

Is it possible to join all the files with input1 based on 1st column? input1 a b c d e f input2 a b input3 a e input4 c (2 Replies)
Discussion started by: quincyjones
2 Replies

6. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

7. Shell Programming and Scripting

Join multiple files

Hi there, I am trying to join 24 files (i showed example of 3 files below). They all have 2 columns. The first columns is common to all. The files are tab delimited eg file 1 rs0001 100e-34 rs0003 2.8e-01 rs008 1.9e-90 file 2 rs0001 1.98e-22 rs0004 3.77e-10... (4 Replies)
Discussion started by: fat
4 Replies

8. Shell Programming and Scripting

Join multiple files with filename

Please help, I want to join multiple files based on column 1, and put the missing values as 0. Also the colname in the output should say which file the values came from. FILE1 1 11 2 12 3 13 FILE2 2 22 3 23 4 24 FILE3 1 31 3 33 4 34 FILE1 FILE2 FILE3 1 11 0 31 (1 Reply)
Discussion started by: newbie83
1 Replies

9. Shell Programming and Scripting

Join two files combining multiple columns and produce mix and match output

I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns. Like I have file A 1234,ABCD,23,JOHN,NJ,USA 2345,ABCD,24,SAM,NY,USA 5678,GHIJ,24,TOM,NY,USA 5678,WXYZ,27,MAT,NJ,USA and file B ... (2 Replies)
Discussion started by: mady135
2 Replies

10. Shell Programming and Scripting

Join files on multiple fields

Hello all, I want to join 2 tabbed files on the first 2 fields, and filling the missing values with 0. The 3rd column in each file is constant for the entire file. file1 12658699 ST5 XX2720 0 1 0 1 53039541 ST5 XX2720 1 0 1.5 1 file2 ... (6 Replies)
Discussion started by: sheetalk
6 Replies
PISG(1) 						      General Commands Manual							   PISG(1)

NAME
pisg -- Perl IRC Statistics Generator DESCRIPTION
pisg is a program which takes IRC logfiles and turns the into nice looking stats, which can be amusing to show to the users of your chan- nel. It's quite simple to set up using command line parameters or a configuration file. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`--'). A summary of options is included below. -ch --channel= Set channel name -cc --cchannels= Only do this channel from configuration file, give multiple times to do multiple channels -l --logfile= Log file to parse -o --outfile= Name of HTML file to create -t --tag= Replace %t in --outfile by argument -ma --maintainer= Channel/stats maintainer -f --format= Logfile format -ne --network= IRC network of the channel -d --dir= Analyze all files in this directory. Ignores --logfile. Give multiple times to use multiple directories. -nf --nfiles= Analyze the last n files if used with --dir -p --prefix= Analyze only files prefixed by something in dir. Only works with --dir -cf --cfg opt=value Specify configuration options -co --configfile= Configuration file -mo --moduledir= Directory containing pisg modules -s --silent Suppress output (except error messages) -v --version Show version -h --help Show help SEE ALSO
The pisg manual is located in /usr/share/doc/pisg/. AUTHOR
This manual page was written by Julien Danjou acid@debian.org for the Debian system (but may be used by others). Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version pub- lished by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts. PISG(1)
All times are GMT -4. The time now is 03:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy