Get both common and missing values from multiple files Post: 302902855

Sponsored Content

Top Forums Shell Programming and Scripting Get both common and missing values from multiple files Post 302902855 by Don Cragun on Thursday 22nd of May 2014 01:10:47 PM

05-22-2014

Registered User

I'm confused. You said that you tried:

Code:

awk 'FNR==NR{a[$2]=$3;next}{print $0,a[$2]?a[$2]:"NA"}' file2 file1

to handle five files, but you only gave this script two files?

The code above seems to be using the 2nd field (not the 1st) as the common key between files and using the 3rd field (not the 2nd) as the data for that key. This is probably why this wasn't doing what you wanted for two files.

Your sample output shows empty fields (with a single space field separator) when a field was missing; you said "missing fields should be NULL" (which could be interpreted to mean you want a null string or that you want the literal string "NULL"); but the code above seems to be trying to put out the literal string NA when the data is an empty string, 0, or missing.

So, I can guess that you want one output file with one output column for each input file.

But it isn't clear if you care whether or not the output is sorted on the 1st output column. Do you care if the output is sorted?

And it isn't clear what you want for output if the data in the 2nd field of an input file is missing, is an empty string, or is a numeric string that evaluates to 0. If there is no line in an input file for a 1st field value, do you want the output to be an empty string, the string NULL, or the string NA? If there is a line in an input file with a given 1st field value but there is no second field on that line in the input file, do you want the output to be an empty string, the string NULL, or the string NA? If there is a line in an input file with a given 1st field and the 2nd field is a string of one or more zeros, do you want the output to be an empty string, the string NULL, the string NA, or the string of one or more zeros found on the corresponding input line?

What do you want to use as the output field separator?

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Scan two files and print values missing

Dear Experts, Have been seraching this forum from this morning for my query but dint find hence posting it her... Basically i have two input files BSS and MSS which has a unique string , hence i hav tried and seperated the text to compare frm both files .. Any my present input files look like...

2. Shell Programming and Scripting

Get common lines from multiple files

FileA chr1 31237964 NP_001018494.1 PUM1 M340L chr1 31237964 NP_055491.1 PUM1 M340L chr1 33251518 NP_037543.1 AK2 H191D chr1 33251518 NP_001616.1 AK2 H191D chr1 57027345 NP_001004303.2 C1orf168 P270S FileB chr1 ...

3. Shell Programming and Scripting

Parsing common values across multiple files

Hi All, I have multiple (5+) text files with single columns and I would like to grep the common values across all the text files and parse it to a new file. All the values are numerical. Please let me know how to do it using awk.

4. UNIX for Dummies Questions & Answers

Extract common data out of multiple files

I am trying to extract common list of Organisms from different files For example I took 3 files and showed expected result. In real I have more than 1000 files. I am aware about the useful use of awk and grep but unaware in depth so need guidance regarding it. I want to use awk/ grep/ cut/...

5. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones...

6. Shell Programming and Scripting

Find Common Values Across Two Files

7. UNIX for Dummies Questions & Answers

Want to change common line from multiple files

Hi everyone, I've a requirement to modify an existing line which is common to multiple files. I need to replace that existing line with a new line. I've almost 900 ksh files to edit in the similar fashion in the same directory. Example: Existing Line: . $HOME/.eff.env (notice the "." at the...

8. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated...

9. Shell Programming and Scripting

Common values in 2 columns in 2 files

Hello, Suppose I have these 2 tab delimited files, where the second column in first file contains matching values from first column of the second file, I would like to get an output like this: File A 1 A 2 B 3 C File B A Apple C Cinnabon B Banana I would like...

10. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions.

LEARN ABOUT HPUX

dh_missing

DH_MISSING(1)							     Debhelper							     DH_MISSING(1)

NAME

       dh_missing - check for missing files

SYNOPSIS

       dh_missing [-Xitem] [--sourcedir=dir] [debhelperoptions]

DESCRIPTION

       dh_missing compares the list of installed files with the files in the source directory. If any of the files (and symlinks) in the source
       directory were not installed to somewhere, it will warn on stderr about that (--list-missing) or fail (--fail-missing).	Please note that
       without either of these options, dh_missing will silently do nothing.

       This may be useful if you have a large package and want to make sure that you don't miss installing newly added files in new upstream
       releases.

       Remember to test different kinds of builds (dpkg-buildpackage -A/-B/...) as you may experience varying results when only a subset of the
       packages are built.

FILES

       debian/not-installed
	   List the files that are deliberately not installed in any binary package.  Paths listed in this file are ignored by dh_missing.
	   However, it is not a method to exclude files from being installed by any of the debhelper tool.  If you want a tool to not install a
	   given file, please use its --exclude option (where available).

	   dh_missing will expand wildcards in this file (since debhelper 11.1).  Wildcards without matches will be ignored.

OPTIONS

       --list-missing
	   Warn on stderr about source files not installed to somewhere.

	   Note that files that are excluded from being moved via the -X option are not warned about.

	   This is the default in compat 12 and later.

       --fail-missing
	   This option is like --list-missing, except if a file was missed, it will not only list the missing files, but also fail with a nonzero
	   exit code.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Michael Stapelberg <stapelberg@debian.org>

11.1.6ubuntu2							    2018-05-10							     DH_MISSING(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Scan two files and print values missing

Discussion started by: shaliniyadav

2. Shell Programming and Scripting

Get common lines from multiple files

Discussion started by: genehunter

3. Shell Programming and Scripting

Parsing common values across multiple files

Discussion started by: Lucky Ali

4. UNIX for Dummies Questions & Answers

Extract common data out of multiple files

Discussion started by: macmath