Sponsored Content
Top Forums Shell Programming and Scripting Compare & print content of file which is not matching Post 302998472 by drl on Friday 2nd of June 2017 09:59:35 AM
Old 06-02-2017
Hi.

Here is an alternative to coding in awk. The crush toolset contains a number of programs to deal with CSV (DSV) files. One such is grepfield.

In this particular case, we need to collect the lines which contain the strings to search for. That's done with paste. After that, it's a simple command line:
Code:
#!/usr/bin/env bash

# @(#) s2       Demonstrate filtering for specific field, crush:grepfield.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C paste grepfield dixf

rm -f f1
E=expected-output.txt

pl " Input data file data[12]:"
head data[12]

pl " File data2 pasted together into variable v1:"
v1="$(paste -d'|' -s data2)"
pe " v1 = $v1"

pl " Expected output:"
cat $E

pl " Results, grep field 4, pipe delimiter:"
# Gathering lines -- from an idea at:
# https://stackoverflow.com/questions/15758814/turning-multiple-lines-into-one-line-with-comma-separated-perl-sed-awk
grepfield -d '|' -v -f 4 "$v1" data1 |
tee f1

pl " Verify results if possible:"
C=$HOME/bin/pass-fail
[ -f $C ] && $C || ( pe; pe " Results cannot be verified." ) >&2

pl " Details on grepfield (one tool in crush suite):"
dixf grepfield

exit 0

producing:
Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.7 (jessie) 
bash GNU bash 4.3.30
paste (GNU coreutils) 8.23
grepfield - ( local: /usr/local/bin/grepfield, 2017-05-17 )
dixf (local) 1.49

-----
 Input data file data[12]:
==> data1 <==
A|B|C|167|D|E
C|K|D|123|D|E
A|B|D|789|G|F
C|D|G|143|A|B

==> data2 <==
123
456
780
143

-----
 File data2 pasted together into variable v1:
 v1 = 123|456|780|143

-----
 Expected output:
A|B|C|167|D|E
A|B|D|789|G|F

-----
 Results, grep field 4, pipe delimiter:
A|B|C|167|D|E
A|B|D|789|G|F

-----
 Verify results if possible:

-----
 Comparison of 2 created lines with 2 lines of desired results:
 Succeeded -- files (computed) f1 and (standard) expected-output.txt have same content.

-----
 Details on grepfield (one tool in crush suite):
grepfield       looks for patterns within a specific field of a flat-file (man)
Path    : /usr/local/bin/grepfield
Version : - ( local: /usr/local/bin/grepfield, 2017-05-17 )
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYSV ...)
Help    : probably available with -h,--help
Home    : http://crush-tools.googlecode.com (doc)

Best wishes ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do I search first&second string & copy all content between them to other file?

Hi All, How do I search first string & second string and copy all content between them from one file to another file? Please help me.. Thanks In Advance. Regards, Pankaj (12 Replies)
Discussion started by: pankajp
12 Replies

2. Shell Programming and Scripting

compare two columns of different files and print the matching second file..

Hi, I have two tab separated files; file1: S.No ddi fi cu o/l t+ t- 1 0.5 0.6 o 0.1 0.2 2 0.2 0.3 l 0.3 0.4 3 0.5 0.8 l 0.1 0.6 ... (5 Replies)
Discussion started by: vasanth.vadalur
5 Replies

3. UNIX for Dummies Questions & Answers

How to compare 2 files & get specific value & replace it in other file.

Hiiii Friends I have 2 files with huge data. I want to compare this 2 files & if they hav same set of vales in specific rows & columns i need to get that value from one file & replace it in other. For example: I have few set data of both files here: a.dat: PDE-W 2009 12 16 5 29 11.11 ... (10 Replies)
Discussion started by: reva
10 Replies

4. Shell Programming and Scripting

compare & print lines with 2 requirements

1.compare 90 logs >20 print off with correct figure 2.compare latest log i.e ciscoresets_20120314 against all records not just the 90 (as above) and any lines not matching print as: NEW:SYDGRE04,10,9 1 (note this number could be 2 or whatever not necessary 1 it could be 2,3 or even 10... (2 Replies)
Discussion started by: slashbash
2 Replies

5. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

6. Shell Programming and Scripting

Pattern Matching & replacing of content in file1 with file2

I have file 1 & file 2 with content mentioned below. I want to get the output as shown in file3. Requirement: check the content of column 1 & column 2, if value of column 1 in file1 matches with first column of file2 then remaining columns(2&3) of file2 should get replaced, also if value of... (4 Replies)
Discussion started by: siramitsharma
4 Replies

7. UNIX for Dummies Questions & Answers

How to append portion of a file content to another file when a certain pattern is matching?

Hi ladies and gentleman.. I have two text file with me. I need to replace one of the file content to another file if one both files have a matching pattern. Example: text1.txt: ABCD 1234567,HELLO_WORLDA,HELLO_WORLDB DCBA 3456789,HELLO_WORLDE,HELLO_WORLDF text2.txt: XXXX,ABCD... (25 Replies)
Discussion started by: bananamen
25 Replies

8. Shell Programming and Scripting

Compare file1 for matching line in file2 and print the difference in matching lines

Hello, I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Discussion started by: RasB15
2 Replies

9. Shell Programming and Scripting

Compare two folders and file content

Hi, I am having two folders where i need to compare the content of files and also to know if any new files been added and redirect the difference output in respective filename logs. For e.g.: Directory D1: f1 f2 f3 Directory D2: f1 f2 f3 f4 i Need to compare the directories... (25 Replies)
Discussion started by: rohit_shinez
25 Replies

10. UNIX for Dummies Questions & Answers

Compare file1 and file2, print matching lines in same order as file1

I want to print only the lines in file2 that match file1, in the same order as they appear in file 1 file1 file2 desired output: I'm getting the lines to match awk 'FNR==NR {a++}; FNR!=NR && a' file1 file2 but they are in sorted order, which is not what I want: Can anyone... (4 Replies)
Discussion started by: pathunkathunk
4 Replies
PASTE(1)						    BSD General Commands Manual 						  PASTE(1)

NAME
paste -- merge corresponding or subsequent lines of files SYNOPSIS
paste [-s] [-d list] file ... DESCRIPTION
The paste utility concatenates the corresponding lines of the given input files, replacing all but the last file's newline characters with a single tab character, and writes the resulting lines to standard output. If end-of-file is reached on an input file while other input files still contain data, the file is treated as if it were an endless source of empty lines. The options are as follows: -d list Use one or more of the provided characters to replace the newline characters instead of the default tab. The characters in list are used circularly, i.e., when list is exhausted the first character from list is reused. This continues until a line from the last input file (in default operation) or the last line in each file (using the -s option) is displayed, at which time paste begins selecting characters from the beginning of list again. The following special characters can also be used in list: newline character tab character \ backslash character Empty string (not a null character). Any other character preceded by a backslash is equivalent to the character itself. -s Concatenate all of the lines of each separate input file in command line order. The newline character of every line except the last line in each input file is replaced with the tab character, unless otherwise specified by the -d option. If '-' is specified for one or more of the input files, the standard input is used; standard input is read one line at a time, circularly, for each instance of '-'. EXIT STATUS
The paste utility exits 0 on success, and >0 if an error occurs. EXAMPLES
List the files in the current directory in three columns: ls | paste - - - Combine pairs of lines from a file into single lines: paste -s -d ' ' myfile Number the lines in a file, similar to nl(1): sed = myfile | paste -s -d ' ' - - Create a colon-separated list of directories named bin, suitable for use in the PATH environment variable: find / -name bin -type d | paste -s -d : - SEE ALSO
cut(1), lam(1) STANDARDS
The paste utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. HISTORY
A paste command appeared in Version 32V AT&T UNIX. BSD
June 25, 2004 BSD
All times are GMT -4. The time now is 04:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy