awk Array matching and replacing from master file.
I have an awk related question that I was hoping you all could help with.
I am given 2 input files named OLDFILE and NEWFILE, and a Master file named MASTERFILE. They can be seen below.
OLDFILE:
Code:
a
a
a
a
a
f
g
4
5
7
8
1
2
3
NEWFILE:
Code:
a
b
c
d
e
f
g
1
2
3
4
5
7
8
MASTERFILE:
Code:
4
5
7
8
1
6
2
3
There is going to a bunch of input values here, and below the letters, and above the numbers.
the blocks of text being changed represent
a very small portion of the entire actual MASTERFILE.
a
a
a
a
a
f
g
Using the OLDFILE and the NEWFILE, the goal is to change/append the MASTERFILE in a way such that any time a line in the MASTERFILE matches with a line in the OLDFILE, that line is replaced with the corresponding line from the NEWFILE.
for example. the first line of the MASTERFILE is 4. 4 is in the OLDFILE, so 4 in the MASTERFILE should be replaced with 1 (correspond value from NEWFILE) in the output.
Here is the code I have written thus far to solve this problem:
Code:
awk '
BEGIN {
while ((getline name < "OLDFILE") > 0)
{
getline name2 < "NEWFILE"
a[name]=name2
}
}
a[$0]{
$0=a[$0]
}1
' MASTERFILE > NEW-MASTERFILE
Currently the output (NEW-MASTERFILE) I am getting is:
Code:
1
2
3
4
5
6
7
8
There is going to a bunch of input values here, and below the letters, and above the numbers.
the blocks of text being changed represent
a very small portion of the entire actual MASTERFILE.
e
e
e
e
e
f
g
The output I am hoping to get is:
Code:
1
2
3
4
5
6
7
8
There is going to a bunch of input values here, and below the letters, and above the numbers.
the blocks of text being changed represent
a very small portion of the entire actual MASTERFILE.
a
b
c
d
e
f
g
This is not the actual code I am going to be using the script on, and I am hoping to make the code as repeatable as possible because it needs to be used thousands of times. Any help that could be provided would be greatly appreciated.
I still see the same problem I saw in your old thread, but with complete input can articulate it better. You will end up with g g g g g g g, not a b c d e f g, because the lines aren't unique -- you can't tell those a's apart. If there happen to be a's you don't want to change, they will all become g anyway.
To do this reliably you need something like diff's output, it actually tells you which line numbers need to change.
Code:
$ diff OLDFILE NEWFILE
$ diff OLDFILE NEWFILE
2,5c2,5
< a
< a
< a
< a
---
> b
> c
> d
> e
7a8,10
> 1
> 2
> 3
12,14d14
< 1
< 2
< 3
$
In fact if you could convince whatever's giving you this output to use diff, you could pretty much forget about this problem. diff generates and applies(via patch) text changes reliably and efficiently. That's what it's there for.
Code:
$ diff OLDFILE NEWFILE > new.patch # Save differences between two files
$ patch OLDFILE new.patch # Edit OLDFILE given the differences listed in new.patch
$ diff OLDFILE NEWFILE
$ echo $? # If they are perfectly identical, diff will return 0
0
$
Yes, I know, editing OLDFILE isn't what you want to do, I'm just showing how you could use diff to replace your ad-hoc editing system. I don't see any reliable way to edit MASTERFILE with the input you have unless you can absolutely guarantee the lines in all three files are all completely, 100% unique. This will be extremely dangerous otherwise.
Last edited by Corona688; 08-14-2014 at 02:20 PM..
In the below awk I am splitting $7 on the : and then counting each line or NM_xxxx. If the $1 value is the same for each line then print the $7 that occurs the most with the matching $1 value. The awk seems close but I am not sure what is going on. I included a description as well as to what I... (1 Reply)
Here is the task that I was presented with:
I am dealing with about a 10,000 line input deck file for an analysis. About 10 separate blocks of around 25 lines of code each need to be updated in the input deck.
The input deck (deckToChange in the code below) comes with 2 separate files. File 1... (5 Replies)
I have an application desigend in PHP and MySQl running on apache web server that I is running on a Amazon EC2 server Centos. I want to implement the master-master and master slave replication and high availability disaster recovery on this application database.
For this I have created two... (0 Replies)
Dear Unix Forums,
I am hoping you can help me with a pattern matching problem.
What am I trying to do?
I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
I need to write an Unix script to report the number of SQL files in my home directory, ending with .sql . The script should, also, be checking whether there is a file with an underscore in its name. If that is the case, the underscore should be converted to a dash (‐); for example... (1 Reply)
Experts,
I am a beginner to Unix Shell Scripting
We have source as a flat file which contains CTRL+F character as the delimiter. We need to count the number of records in the file (CTRL+F) to perform file validation
Following command being used:
awk '{cnt+=gsub(//,"&")}END {print cnt}'... (4 Replies)
Im new to using perl scripting, and i was wondering if anyone could help me, I need to create a cgi file that when it runs it opens a file looks through all the words, and replace the word hello with goodbye, ive been looking around and someone said to use an If statement, but ive found no other... (2 Replies)
Hi all
I need to replace all characters in a file except for the matching pattern
eg.
I need to replace all character with '*' except for the pattern "abc"
Input
"sdfhgsdf abc ##%$#abcsdfh sdfjkfff"
Output
"******abc******abc*************"
Request for single liner solution (1 Reply)
I have a set of user name password pairs in one file and I have a set of files which have user names and a random string as the password. How can I read the user name and password from the first file and replace the random password in the second file.
my list file looks like this
user1, pass1... (2 Replies)
I would like to find a list of files in a directory less than 2 days old and put them into an array variable. And then search for each file in the array for a matching string say "Return-code= 0". If it matches, then display the array element with a message as "OK".
Your help will be greatly... (1 Reply)
