Shell Programming and Scripting

I have two files. One is consisting of one line, with data separated by spaces and each number appearing only once.
The other is consisting of one column and multiple lines which can have some numbers appearing more than once.
It looks something like this:

file 1:

Code:

20 700 15 30

file2:

Code:

10
10 
200
200
700
700
700
20
30
30
50

(The files are a result of some other processing and scripts so there could be some extra spaces or tabs that I cannot easily influence/remove)

I would like to print the lines from file2 that do not have a match in file1. It is very important that in case there aren't any lines in file2 that do not have a match in file1 (i.e. when the file2 doesn't contain any numbers that aren't already in file1), I get a completely empty file, and not spaces or any other characters.

I have found some ways to do it when both files are columns, but not when one of them is a one line. When I tried transforming the one line file into a one column file, I got some unwanted spaces in the output.

Thank you!

Moderator's Comments:

Please use CODE tags as required by forum rules!

Welcome to the forum.


Please show the attempts you made and where you got stuck.

I tried with turning file1 into a column file with:

Code:

tr ' ' '\n' < file1 | awk '{print $0}' > file1_new

and then solving it by working with columns

Code:

awk '{k = $1} NR==FNR{a[k]; next} !(k in a)' file1_new  file2

However, I then got an empty line as the output (instead of the wanted empty file) when both files contained the same numbers (as described in the end of my original post). I would like to solve it without modifying file1, but I don't know how to approach and start there.

comm -23 will work for you

thanks I've tried that now but I still have the same problem as when using the code from my second post.
With the files containing different numbers as in my first post, I get empty lines as first and last line.



Since my data is not in columns but in one line for file1, and they are a part of a cshell script and come as results in a loop,it would be difficult to be sure that it will never have any extra characters, I would rather keep them as a one line instead of converting to a column.

Is there a way to use indices with lines as with the columns in awk?

You don't need to change the original files, but you can do whatever is needed to your own work files.

You were getting close by breaking many numbers on one line to one per line. From there, sort copies of both files the same way (file 2 may need a unique sort) and then run them through diff. This process won't work if diff outputs "c" lines with both "<" and ">", but if not you can take out lines containing a or d, then take out the first two characters of all other lines. For example:

Code:

diff file1 file2 |grep -v d |  sed 's/..//' >outputfile

Sorry I don't understand what do you mean by c lines and lines containing a or d?


Also, this code gave me the which has in its second line two numbers separated by comma which are in neither of the files, is that some counter of data entries?