Print lines present in first that match second variable. And ones that does not exist in second.
I have multi line input(var1) and reference(var2) variables.
How to capture lines not present in var2 but present in var1?
How to capture lines present var2 but not in var1?
Code:
# configuration from server
var1="""
Custom JAX-RS
Custom Shared
Web 2.0
"""
# required configuration
var2="""
Web 2.0
Custom Shared
IBM WebSphere JAX-RS
"""
# matching ones between var1 and var2
valid=''
# capture lines not present in var2 but present in var1?
invalid=''
# capture lines present in var2 but not in var1?
missing=''
I have this working by using two for loops(one to find invalid/valid and the other to find missing) and some if statements. I am not posting to avoid confusion and clutter.
I am sure there is a better way of doing this using awk or perl instead of writing basic loops and grep statements.
Last edited by kchinnam; 08-29-2016 at 11:49 PM..
Reason: updating requirements
With almost 200 posts to your credit, we would hope that you would have learned enough from the suggestions you've seen posted here that you would have a good start at getting the results you want. What have you tried to solve this problem on your own?
What operating system and shell (including version number) are you using?
With five lines of text (two of which are empty lines) in var1 and three lines of text in var2, what output are you hoping to produce?
I have GNU bash, version 3.2, SuSE Linux.
Again I have this working using lengthy basic for loops, grep and if statements. Love to see If there is an efficient way of doing this without lot of loops.
Last edited by kchinnam; 08-29-2016 at 11:48 PM..
Reason: typo
I am not trying to rub anything in. The way we learn how to do things is to try to do it on our own. Then, if we get stuck, ask for help figuring out what doesn't work. Or, if we get something that works, but would like to make it more portable, more efficient, or faster; ask for suggestions for improvements. Study the suggestions you get. If you don't understand how or why it works, look at the manual page on your system for that utility and try to figure it out again. If you still don't understand, ask for an explanation describing how the code you don't understand works.
If you don't try to do something on your own before asking for a solution from someone else, you'll never learn how to do it on yourself. We are here to help you learn how to solve problems; not to act as your unpaid programming staff.
And, it is always a good idea to tell us what operating system and shell you're using. Many utilities have different options and capabilities on different systems. And various shells have various extensions to the features required by the standards.
On a slightly different, but very closely related, topic: How would you approach this problem if the strings you were comparing were in files instead of in shell variables?
And, with the shell you are using, what is the difference between the results produced by the three commands:
Code:
var1="""
Custom JAX-RS
Custom Shared
Web 2.0
"""
var1="
Custom JAX-RS
Custom Shared
Web 2.0
"
var1='
Custom JAX-RS
Custom Shared
Web 2.0
'
This User Gave Thanks to Don Cragun For This Post:
Three quotes are used to assign multi-line text to variable without any interpretations by shell.
Double quotes would cause text to be interpreted(variables expanded) by shell, so it is not a good choice to assign multi line text.
Single quotes is to treat text as is(I never used it to assign multi-line input).
Since this is to verify if pre defined required libs are present from a dump file created from large number of servers, I can't use diff.
I understand you feel that I could have done better.. appreciate it.
Three double quotes will work exactly as one double quote does, as the first and last pair will enclose the empty string; and shell expansions will be done in both. To avoid expansions, use single quotes.
I think chances to get help here are better if you post your own attempts, be they faulty or elegant.
Similar problems have been solved in here umpteen times, c.f. this recent thread and the links in its bottom part.
First consider the following script stored in a file named tester:
Code:
#!/bin/bash
IAm=${0##*/}
var1="""
Custom JAX-RS
$IAm
Custom Shared
$(who am I)
Web 2.0
"""
var2="
Custom JAX-RS
$IAm
Custom Shared
$(who am I)
Web 2.0
"
var3='
Custom JAX-RS
$IAm
Custom Shared
$(who am I)
Web 2.0
'
printf 'var1begin%svar1end\n\nvar2begin%svar2end\n\nvar3begin%sbar3end\n\n' \
"$var1" "$var2" "$var3"
if [ "$var1" = "$var2" ]
then echo '"$var1" is the same as "$var2".'
else echo '"$var1" is not the same as "$var2".'
fi
if [ "$var1" = "$var3" ]
then echo '"$var1" is the same as "$var3".'
else echo '"$var1" is not the same as "$var3".'
fi
if [ "$var2" = "$var3" ]
then echo '"$var2" is the same as "$var3".'
else echo '"$var2" is not the same as "$var3".'
fi
From your description of how """string""" works, the variable expansion $IAm and the command substitution $(who am i) should not occur in the assignments to var1 and var3, but should occur in the assignment to var2. But, as RudiC said and according to the standards and the bash man page, the above construct is a concatenation of an empty string (""), the evaluation of "string" (including parameter expansions, command substitutions, and arithmetic expansions), and another empty string (""). In other words, """string""" produces exactly the same results as
Code:
"string"
no matter how many <newline> characters are contained in string (as long as there are no unquoted double-quote characters in string).
On my system, running the above code produces the output:
Code:
var1begin
Custom JAX-RS
tester
Custom Shared
dwc ttys007 Aug 17 13:55
Web 2.0
var1end
var2begin
Custom JAX-RS
tester
Custom Shared
dwc ttys007 Aug 17 13:55
Web 2.0
var2end
var3begin
Custom JAX-RS
$IAm
Custom Shared
$(who am I)
Web 2.0
bar3end
"$var1" is the same as "$var2".
"$var1" is not the same as "$var3".
"$var2" is not the same as "$var3".
Note also that all of the assignments in the above script produce an empty line as the first line in each of the assignments to var1, var2, and var3. If you don't want to include an empty line and don't want variable expansions, command substitutions, or arithmetic expansions to be performed in a variable intended to contain three lines, you should use single quotes like this:
Moving on. As RudiC said, this question has been asked many times before, except that your input is in shell variables instead of in files. (Although you have not given any reason that makes sense to me as to why you are hard-coding the values of these variables into strings instead of the storing the output produced on the various servers and your expected configuration values in files instead.) The diff utility would not be a way to compare two files unless all lines that you are comparing are guaranteed to always appear in the same order if they are present. But, you might want to consider:
But, grep won't work the way you need it to when you have empty lines in one or both of your files (and you have an empty line as the 1st line in both of your variables).
But what you want can be done with one invocation of awk:
Code:
#!/bin/bash
# configuration from server
var1='
Custom JAX-RS
Custom Shared
Web 2.0
'
# required configuration
var2='
Web 2.0
Custom Shared
IBM WebSphere JAX-RS
'
# matching ones between var1 and var2
printf '%s,\n%s' "$var1" "$var2" | awk -v sq="'" '
BEGIN { printf("valid=%s", sq)
}
n { v2[$0]
if($0 in v1) {
printf("%s%s", sep, $0)
sep = ","
}
next
}
/^,$/ { n = 1
next
}
{ v1[$0]
}
END { printf("%s\ninvalid=%s", sq, sq)
sep = ""
for(i in v1)
if(!(i in v2)) {
printf("%s%s", sep, i)
sep = ","
}
sep = ""
printf("%s\nmissing=%s", sq, sq)
for(i in v2)
if(!(i in v1)) {
printf("%s%s", sep, i)
sep = ","
}
print sq
}'
But the order in which input lines appear in the output may vary with different versions of awk. (And, note that the leading comma in the valid= line indicates that there is an empty line in both $var1 and $var2.)
This User Gave Thanks to Don Cragun For This Post:
I have input file as below I need to check for a pattern and if it is there in file then I need to print all the lines below BEGIN and END keyword. Could you please help me how to get this in AIX using sed or awk.
Input file:
ABC
******** BEGIN *****
My name is Amit.
I am learning unix.... (8 Replies)
Hello,
I have a file contains two columns. I need to print the lines after “xxx” so i'm trying to match "xxx" & cut the lines after that. I'm trying with the grep & cut command, if there any simple way to extract this please help me.
Sample file :
name id
AAA 123
AAB 124
AAC 125... (4 Replies)
URGENT HELP IS NEEDED!!
I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but
- Currently, script is not printing lines to File3.txt in order.
- Also the matching lines are not moving out of File1.txt
... (1 Reply)
I am getting the varible value from a grep command as:
var=$(grep "Group" File1.txt | sed 's/Group Name*//g;s/,//g;s/://g;s/-//g')
which leaves me the value of $var=xyz.
now i want to append $var value in the begining of all the lines present in the file. Can u please suggest?
Input file:
1... (10 Replies)
I need to print the lines that do not match a pattern. I tried using grep -v and sed -n '/pattern/!p', but both of them are not working as I am passing the pattern as variable and it can be null some times.
Example
........ abcd......
.........abcd......
.........abcd......... (4 Replies)
Hi, I have this file.
close
block3c
block3b
block3a
open
close
block2b
block2a
open
close
block1a
open
and I need :
open
block3a
block3b
block3c
close (1 Reply)
I am using Solaris, I want to print
3 lines before pattern match
pattern
5 lines after pattern match
Pattern is abcd to be searched in a.txt. Looking for the solution in sed/awk/perl. Thanks ..
Input File a.txt:
=================
1
2
3
abcd
4
5
6
7
8 (7 Replies)
Ok so I can use awk to match a pattern and print the whole line with print $0. Is there any way to just tell awk to print every line of output when the pattern matches?
I'm having it wait for the word error and then print that entire line. But what I actually need to see is all the following... (9 Replies)
I am trying to parse iostat output for io issues..
I want to print all lines including second occurance of 'extended' till EOF(end of file). Can we do that using awk or sed one liners or do we need a script for it? (1 Reply)
08-29-2016
