The UNIX and Linux Forums  


Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Merge lines into one anypager Shell Programming and Scripting 7 05-22-2008 04:20 PM
merge multiple lines from flat file hnhegde Shell Programming and Scripting 4 12-05-2006 07:13 PM
Use sed to merge multiple lines xb88 Shell Programming and Scripting 3 08-08-2006 02:54 PM
Merge wrapped lines braindrain Shell Programming and Scripting 1 07-25-2005 08:03 PM
Omit repeating lines TheCrunge UNIX for Dummies Questions & Answers 6 02-22-2005 06:26 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 09-28-2007
bluemoon1 bluemoon1 is offline
Registered User
  
 

Join Date: Sep 2007
Posts: 53
merge 2 files (without repeating any lines)

I need to add the content of file1 to file2 - all lines but not those existing in file2 already, so the "cat file1 >> file2" doesn't work.

For example,
file1:
100 xxxxxx str1
102 xxxxxx str2


File2:
50 xxxxxxx xxx
30 xxxxxxxxxxx
102 xxxxxx str2 xxxx
......

the result:
50 xxxxxxx xxx
30 xxxxxxxxxxx
102 xxxxxx str2 xxxx
.....
100 xxxxxx str1

Also, the second line in file1 & third in file2 can either be completely same or with the same patern - starting with a same string and having another same string anywhere in the line.

Please help!
Thank you so much.
(it's a bourne-sh)
  #2 (permalink)  
Old 09-28-2007
summer_cherry summer_cherry is offline Forum Advisor  
Registered User
  
 

Join Date: Jun 2007
Location: Beijing China
Posts: 1,089
cat sort uniq

Hi,
I think you can use cat first to join these two files together, then sort them, and then use the uniq to delete the duplicated lines.

input:

Code:
a>
1
2
3
4
5
6
b>
3
1
2
3
5
342
45
234
2
3

output:

Code:
1
2
234
3
342
4
45
5
6

code:

Code:
cat a>>b | cat b | sort | uniq

  #3 (permalink)  
Old 09-28-2007
bluemoon1 bluemoon1 is offline
Registered User
  
 

Join Date: Sep 2007
Posts: 53
Hi summer_cherry

Theoretically, this would work. However, since the files involved are system config files, with certain lines grouped together for a reason, so I'd hate to sort them. BTW, it also means all the empty lines or empty lines starting # between sections would be gone- as they would be meaningless anyway.

I was thinking to circle through the file(2) for each line (from f1) to be addded -but surely there'd be a better solution (& I don't know enough about the utilities to figure it out)

Thank you.
  #4 (permalink)  
Old 09-29-2007
varungupta varungupta is offline
Registered User
  
 

Join Date: Feb 2007
Location: Pune, Dehradun (INDIA), Michigan(US)
Posts: 206
Quote:
Originally Posted by bluemoon1 View Post
I need to add the content of file1 to file2 - all lines but not those existing in file2 already, so the "cat file1 >> file2" doesn't work.

For example,
file1:
100 xxxxxx str1
102 xxxxxx str2


File2:
50 xxxxxxx xxx
30 xxxxxxxxxxx
102 xxxxxx str2 xxxx
......

the result:
50 xxxxxxx xxx
30 xxxxxxxxxxx
102 xxxxxx str2 xxxx
.....
100 xxxxxx str1

Also, the second line in file1 & third in file2 can either be completely same or with the same patern - starting with a same string and having another same string anywhere in the line.

Please help!
Thank you so much.
(it's a bourne-sh)
Hey,

You can get the first line from the first file and find is it there in second file ? If yes then dont append otherwise append the line in the second file.

YOu can use grep to search the first line from first file and awk to use the same line as pattern to be searched in the second file.
  #5 (permalink)  
Old 10-01-2007
bluemoon1 bluemoon1 is offline
Registered User
  
 

Join Date: Sep 2007
Posts: 53
varungupta:
This is actually what I'm doing now- I had some special key words added in a comment line indicating the start of the file1 & before adding anything I check file2 if that line exists already or not - however I don't think it is safe, as nothing prevents that line from getting deleted in file2 over time...

summer_cherry's solution would work perfectly however re-ordering the file is not acceptible.

Perhaps I need a merge function, to do a diff and add lines from file1 that are not in file2?

It'd be tricky though how to remove the lines that have been added from file2 later.

Any thoughts would be appreciated!
  #6 (permalink)  
Old 10-01-2007
porter porter is offline Forum Advisor  
Registered User
  
 

Join Date: Jan 2007
Posts: 2,965
Quote:
Originally Posted by bluemoon1 View Post
Any thoughts would be appreciated!
It is good practice to store configurations files that you change in RCS somewhere, so if you need to backtrack or do problem determination you can refer to previous revisions.
  #7 (permalink)  
Old 10-05-2007
bluemoon1 bluemoon1 is offline
Registered User
  
 

Join Date: Sep 2007
Posts: 53
Hi porter,
I do back up all the config files during the installation. The thing is, I can't simply restore those files during the uninstall. This is because of the fact that some of the files may have been changed over the time. The customers would have to lose their data if we restore the original files. If at the unisntall time we remove exactly what we added during the install, then it'd be safe.

Bluemoon
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 05:51 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0