Sponsored Content
Top Forums Shell Programming and Scripting Compare columns of multiple files and print those unique string from File1 in an output file. Post 302864081 by drl on Tuesday 15th of October 2013 10:51:09 PM
Old 10-15-2013
Hi.

Using grep, assuming that the 24 files will fit into memory available:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate inverse, "-v", match, grep with auxiliary file.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C grep

pl " Input data file primary data*:"
head primary data*

pl " Results:"
grep -v -f <( cat data* ) primary

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
grep GNU grep 2.5.3

-----
 Input data file primary data*:
==> primary <==
123abc
456def
789ghi

==> data1 <==
123abc
456def
891jkl

==> data2 <==
234mno
123abc
456def

-----
 Results:
789ghi

See man pages for details.

Best wishes ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

compare columns from seven files and print the output

Hi guys, I need some help to come out with a solution . I have seven such files but I am showing only three for convenience. filea a5 20 a8 16 fileb a3 42 a7 14 filec a5 23 a3 07 The output file shoud contain the data in table form showing first field of... (7 Replies)
Discussion started by: smriti_shridhar
7 Replies

2. Shell Programming and Scripting

Compare multiple fields in file1 to file2 and print line and next line

Hello, I have two files that I need to compare and print out the line from file2 that has the first 6 fields matching the first 6 fields in file1. Complicating this are the following restrictions 1. file1 is only a few thousand lines at most and file2 is greater than 2 million 2. I need to... (7 Replies)
Discussion started by: gillesc_mac
7 Replies

3. Shell Programming and Scripting

compare two columns of different files and print the matching second file..

Hi, I have two tab separated files; file1: S.No ddi fi cu o/l t+ t- 1 0.5 0.6 o 0.1 0.2 2 0.2 0.3 l 0.3 0.4 3 0.5 0.8 l 0.1 0.6 ... (5 Replies)
Discussion started by: vasanth.vadalur
5 Replies

4. Programming

Compare two files of 4 columns and o/p unique,append zero's

I have to files File1 1 23 2 34 3 7 4 56 5 61 6 22 7 65 File2 2 21 4 32 7 22 Now i need to compare column1 of both the files and generate a third file which should contain all the values of 1st column of 1st file and in the second column i need to get the coressponding row... (2 Replies)
Discussion started by: kamuju
2 Replies

5. Shell Programming and Scripting

awk compare specific columns from 2 files, print new file

Hello. I have two files. FILE1 was extracted from FILE2 and modified thanks to help from this post. Now I need to replace the extracted, modified lines into the original file (FILE2) to produce the FILE3. FILE1 1466 55.27433 14.72050 -2.52E+03 3.00E-01 1.05E+04 2.57E+04 1467 55.27433... (1 Reply)
Discussion started by: jm4smtddd
1 Replies

6. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

7. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

8. Shell Programming and Scripting

[Solved] awk compare two different columns of two files and print all from both file

Hi, I want to compare two columns from file1 with another two column of file2 and print matched and unmatched column like this File1 1 rs1 abc 3 rs4 xyz 1 rs3 stu File2 1 kkk rs1 AA 10 1 aaa rs2 DD 20 1 ccc ... (2 Replies)
Discussion started by: justinjj
2 Replies

9. UNIX for Dummies Questions & Answers

Compare file1 and file2, print matching lines in same order as file1

I want to print only the lines in file2 that match file1, in the same order as they appear in file 1 file1 file2 desired output: I'm getting the lines to match awk 'FNR==NR {a++}; FNR!=NR && a' file1 file2 but they are in sorted order, which is not what I want: Can anyone... (4 Replies)
Discussion started by: pathunkathunk
4 Replies

10. UNIX for Beginners Questions & Answers

Split into multiple files by using Unique columns in a UNIX file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|1|1 123|12|5679|BCDEFG|34CD56|2019-07-10|2019-07-10|896.7200|1|2... (3 Replies)
Discussion started by: RVSP
3 Replies
gfs2_tool(8)						      System Manager's Manual						      gfs2_tool(8)

NAME
gfs2_tool - interface to gfs2 ioctl/sysfs calls SYNOPSIS
gfs2_tool COMMAND [OPTION]... DESCRIPTION
gfs2_tool is an interface to a variety of the GFS2 ioctl/sysfs calls. Some of the functions of gfs_tool have been replaced by standard sys- tem tools such as mount and chattr, so gfs2_tool doesn't have as many options as gfs_tool used to. COMMANDS
clearflag Flag File1 File2 ... Clear an attribute flag on a file. This is now obsolete and kept only for backward compatibility, chattr is the preferred way to clear attribute flags. See setflag for available flags. This option will probably be removed at a future date. freeze MountPoint Freeze (quiesce) a GFS2 cluster. gettune MountPoint Print out the current values of the tuning parameters in a running filesystem. A better source of similar (more comprehensive) information is that in the /proc/mounts file. Running the mount command with no arguments will also provide the same information. This option is considered obsolete and will probably be removed at some future date. journals MountPoint Print out information about the journals in a mounted filesystem. lockdump MountPoint Print out information about the locks this machine holds for a given filesystem. This information is also available via the debugfs glock dump file, and accessing that file is the preferred method of obtaining a dump of the glock state. sb device proto [newvalue] View (and possibly replace) the name of the locking protocol in the file system superblock. The file system shouldn't be mounted by any client when you do this. sb device table [newvalue] View (and possibly replace) the name of the locking table in the file system superblock. The file system shouldn't be mounted by any client when you do this. sb device ondisk [newvalue] View (and possibly replace) the ondisk format number in the file system superblock. The file system shouldn't be mounted by any client when you do this. No one should have to use this. sb device multihost [newvalue] View (and possibly replace) the multihost format number in the file system superblock. The file system shouldn't be mounted by any client when you do this. No one should have to use this. sb device uuid [newvalue] View (and possibly replace) the uuid in the file system superblock. The file system shouldn't be mounted by any client when you do this. The new uuid value should be in the standard uuid format. For example: 1AEA8269-15C5-72BD-6D83-8720B17AA4EE sb device all Print out the superblock. setflag Flag File1 File2 ... Set an attribute flag on a file. The currently supported flags are jdata, immutable, appendonly, noatime, and sync. The chattr command is the preferred way to set attributes on files. This option will probably be removed at a future date. The jdata flag causes all the data written to a file to be journaled. If the jdata flag is set for a directory, all files and directories subsequently created within that directory are also journaled. This behavior replaces the old inherit_jdata flag from gfs. Same as chattr +j. The immutable flag marks the file immutable. The behavior is similar to the immutable flag in the ext2/3 filesystems. All write access is denied. Same as chattr +i. The appendonly flag causes all data to be written at the end of the file. Same as chattr +a. The noatime flag disables updates to the file's access time. Same as chattr +A. The sync flag causes data written to the file to be sync'ed to stable storage immediately. Same as chattr +S. settune MountPoint parameter newvalue Set the value of tuning parameter. Use gettune for a listing of tunable parameters. The mount -oremount command is the preferred way to set the values of tunable parameters. At some future stage, when all parameters can be set via mount, this option will be removed. unfreeze MountPoint Unfreeze a GFS2 cluster. version Print out the version of GFS2 that this program goes with. withdraw MountPoint Cause GFS2 to abnormally shutdown a given filesystem on this node. This feature is only useful for testing and should not be used during normal filesystem operation. gfs2_tool(8)
All times are GMT -4. The time now is 11:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy