Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Merge files with multi-line stanzas

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Merge files with multi-line stanzas
# 1  
Old 08-09-2016
Merge files with multi-line stanzas
Hello.
I try to merge few /etc/qconfig files in AIX (from few servers into one file on new server).
Could you please help me what would be the best way to merge files that contain multi-line stanzas, like this:

Code:
stanza1:
  attr1 = value1
  attr2 = value2
  attr3 = value3
stanza2:
  attr1 = value1
stanza3:
  attr1 = value1
  attr2 = value2

Thanks in advance.
Last edited by Scrutinizer; 08-09-2016 at 02:48 PM.. Reason: code tags
# 2  
Old 08-09-2016
Without knowing what you want the output to be after you merge (what appears to be a single file) it is hard to explain what you need to do. Or have you shown us the output you want but not shown us the input files that you want to merge to produce that output?

For us to help you do what you want to do we need sample input files (shown in CODE tags) and the corresponding output that should be produced by merging those input files (again shown in CODE tags).
This User Gave Thanks to Don Cragun For This Post:
drl
# 3  
Old 08-10-2016
Ok, sorry, I wasn't to clear. I will give example on real files I'm trying to merge. I'll paste only parts of files, as an example.

Source file 1:

Code:
62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101

Source file 2:

Code:
pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
oss-prnsrv2_3:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9102
oss-prnsrv2_1:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100


Target file should be like this:

Code:
62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
oss-prnsrv2_3:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9102
oss-prnsrv2_1:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100


Thanks for any help.
# 4  
Old 08-11-2016
I don't understand what the rules are for your merge. You say that the target file should contain two stanzas wth the heading hp@Jet133:, two stanzas with the heading hp@JetNL95:, and three stanzas with the heading hp@JetNL97:.

I would have thought that the purpose of merging stanzas would be to get rid of stanzas with the same name. Since that clearly is not your requirement, please explain the rules you want to use to determine which stanzas are to be copied into the result file and which stanzas are to be dropped.
This User Gave Thanks to Don Cragun For This Post:
drl
# 5  
Old 08-11-2016
Ok, more explanation from me.
File structure is - two-line stanza defining a print queue (which refers to a device) and after that multi-line stanza defining this device. In examples above I have a situation, that multi print queues use the same device - that's why device definition is duplicated.
After I've made some checks - we can get rid of additional device definitions, so correct target file should be like:

Code:
62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9101
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9101
62_133_3_zebra:
        device = hp@Jet133
oss-prnsrv2_3:
        device = hp@JetNL97
oss-prnsrv2_1:
        device = hp@JetNL97
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

# 6  
Old 08-11-2016
This still is far from clear and complete. What should happen if there are several different "stanzas" for the same device, as for hp@JetNL97, that points to ports 9100, 9101, and 9102? And, are there any requirements on the order of output lines?
This is based an some assumptions on my side; it keeps the last device definition encountered and scraps the earlier ones. Plus, it has been developed and tested on my linux system; it may need some adaptions to run on your AIX. Still, I think it comes close to what you posted as the desired output:

Code:
awk '
!/^ /           {IX = $0
                 if (!(IX in TMP)) NIX[++CNT] = IX
                 delete TMP[IX]
                 next
                }
                {TMP[IX] = TMP[IX] RS $0
                }
END             {for (i=1; i<=CNT; i++) print NIX[i]  TMP[NIX[i]]
                }
' file[12]
62_it_91_1_las_ps:
        device = hp@Jetit01
hp@Jetit01:
        file = /var/spool/lpd/pio/@local/dev/hp@Jetit01#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jetit01 9100
62_133_3_lbl:
        device = hp@Jet133
hp@Jet133:
        file = /var/spool/lpd/pio/@local/dev/hp@Jet133#hpJetDirect#9102
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd Jet133 9102
testnl:
        device = hp@JetNL95
hp@JetNL95:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL95#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL95 9100
pnr5:
        device = hp@JetNL95
oss-prnsrv2_2:
        device = hp@JetNL97
hp@JetNL97:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL97#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL97 9100
62_133_3_zebra:
        device = hp@Jet133
oss-prnsrv2_3:
        device = hp@JetNL97
oss-prnsrv2_1:
        device = hp@JetNL97
oss-prnsrv03_1:
        device = hp@JetNL98
hp@JetNL98:
        file = /var/spool/lpd/pio/@local/dev/hp@JetNL98#hpJetDirect#9100
        header = never
        trailer = never
        access = both
        backend = /usr/lib/lpd/pio/etc/piojetd JetNL98 9100

Last edited by RudiC; 08-11-2016 at 08:06 AM.. Reason: Removed one <TAB> from code and output.
# 7  
Old 08-11-2016
Keeping the 1st occurrence of stanzas with the same first line produces the requested output and makes the code a little simpler:
Code:
awk '
!/^[ \t]/ {
	copy = !($1 in stanza)
	stanza[$1]
}
copy' file[12]

If someone else wants to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge multi-lines into one single line using shell script or Linux command

Hi, Can anyone help me for merge the following multi-line log which beginning with a " and line ending with ": into one line. *****Original Log***** 087;2008-12-06;084403;"mc;;SYHLR6AP1D\LNZW;AD-703;1;12475;SYHLR6AP1B;1.1.1.1;0000000062;HGPDI:MSISDN=12345678,APNID=1,EQOSID=365;... (3 Replies)
Discussion started by: rajeshlinux2010
3 Replies

2. Shell Programming and Scripting

Multi line log files to single line format

I want to read the log file which was generate from other command . And the output was having multi line in log files for job name and server name. But i need to make all the logs on one line Source file 07/15/2018 17:02:00 TRANSLOG_1700 Server0005_SQL ... (2 Replies)
Discussion started by: ranjancom2000
2 Replies

3. Shell Programming and Scripting

Take out First Line and merge all files

Hi Gurus, I have n number of files. Data which is in the files have column headers. I need to take them out and merge into one file. Can you help please? I need to do that little urgent. Thanks (4 Replies)
Discussion started by: raopatwari
4 Replies

4. Shell Programming and Scripting

Multi-line filtering based on multi-line pattern in a file

I have a file with data records separated by multiple equals signs, as below. ========== RECORD 1 ========== RECORD 2 DATA LINE ========== RECORD 3 ========== RECORD 4 DATA LINE ========== RECORD 5 DATA LINE ========== I need to filter out all data from this file where the... (2 Replies)
Discussion started by: Finja
2 Replies

5. Shell Programming and Scripting

Merge multi-line output into a single line

Hello I did do a search and the past threads doesn't really solve my issue. (using various awk commands) I need to combine the output from java -version into 1 line, but I am having difficulties. When you exec java -version, you get: java version "1.5.0_06" Java(TM) 2 Runtime... (5 Replies)
Discussion started by: flagman5
5 Replies

6. Shell Programming and Scripting

merge files based on line by line

Hi, lets assume the following details.. file 1 has below details abc|1234|xyz abc1|1234|xyz1 abc2|1234|xyz2 and file 2 has below details pqr|124|lskd ebwer|325|dfb wf|234|sdb I need out put shown below abc|1234|xyz pqr|124|lskd abc1|1234|xyz1 ebwer|325|dfb abc2|1234|xyz2... (4 Replies)
Discussion started by: alnhk
4 Replies

7. Shell Programming and Scripting

Help on Merge multi-lines into one single line

Hello, Can anyone let me know how to use Perl script to Merge following multi-lines into one single line... ***** Multi-line***** FILE_Write root OK Tue Jul 01 00:00:00 2008 cl_get_path file descriptor = 1 FILE_Write root OK ... (5 Replies)
Discussion started by: happyday
5 Replies

8. Shell Programming and Scripting

How to use Perl to merge multi-line into single line

Hi, Can anyone know how to use perl to merge the following multi-line information which beginning with "BAM" into one line. For each line need to delete the return and add a space. Please see the red color line. ******Org. Multi-line) BAM admin 101.203.57.22 ... (3 Replies)
Discussion started by: happyday
3 Replies

9. Shell Programming and Scripting

Merge multi-lines into one single line

Hi, Can anyone help me for merge the following multi-line log which beginning with a number and time: into one line. For each line need to delete the return and add a space. Please see the red color line. *****Original Log*****... (4 Replies)
Discussion started by: happyday
4 Replies

10. Shell Programming and Scripting

merge multi-lines into one line

Hi, Can anyone help me for merge the following multi-line logs( the black lines) which beginning with time: into one line. For the line with "-", it needs to be deleted. Please see the red color line. ######################################### time: 20080817073334 dn: uid=ok,ou=nbt... (3 Replies)
Discussion started by: missyou
3 Replies
Login or Register to Ask a Question