Shell Script to Create non-duplicate lists from two lists


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell Script to Create non-duplicate lists from two lists
# 1  
Old 04-05-2010
Shell Script to Create non-duplicate lists from two lists

File_A contains Strings:
a
b
c
d

File_B contains Strings:
a
c
z



Need to have script written in either sh or ksh. Derive resultant files (File_New_A and File_New_B) from lists File_A and File_B where string elements in File_New_A and File_New_B are listed below.

Resultant files:
File_New_A contains String:
a
c


File_New_B contains String:
z


Thank you in advance for your help.
mlv_99
# 2  
Old 04-05-2010
Shell Script: Compare 2 Files and Extract Unique Elements of Each File into 2 New Files

Thanks for the reply danmero. I made a mistake in the description. Let me re-phrase the question again.


File_A contains Strings and content's elements are NOT IN ANY ORDER OR PATTERN AND file varies in size from one run to the next run:
a <- in both File_A and File_B
b <- unique to File_A
c <- in both File_A and File_B
d <- unique to File_A

File_B contains Strings and content's elements are NOT IN ANY ORDER OR PATTERN AND file varies in size from one run to the next run:
a <- in both File_A and File_B
c <- in both File_A and File_B
z <- unique to File_B



Need to have script written in either sh or ksh. Resultant files (File_New_A and File_New_B) is derived from comparing contents of files File_A and File_B where resultant string elements in File_New_A and File_New_B are listed below. Resultant File_New_A will contain only unique elements found only in File_A when compared with File_B. File_New_B will contain only unique elements found in only File_B when compared with File_A.


Resultant files:
File_New_A contains String:
b <- unique to File_A
d <- unique to File_A


File_New_B contains String:
z <- unique to File_B


++++++++++++++++++++++++++++++++++++++

The next run could have

File_A contains Strings and content's elements are NOT IN ANY ORDER OR PATTERN AND file varies in size from one run to the next run:
m <- only in File_A
n <- only in File_A
o <- only in File_A
p <- only in File_A
q <- in both File_A and File_B
r <- in both File_A and File_B

File_B contains Strings and content's elements are NOT IN ANY ORDER OR PATTERN AND file varies in size from one run to the next run:
q <- in both File_A and File_B
r <- in both File_A and File_B

Resultant files after comparing File_A and File_B:

Resultant files:
File_New_A contains String:
m <- unique to File_A
n <- unique to File_A
o <- unique to File_A
p <- unique to File_A

File_New_B contains String:

No elements in File_New_B since File_B have no unique elements to File_B, all files in File_B are also in File_A; therefore File_New_B is empty.

I hope this is more clearer.

Thank you.

Matt
# 3  
Old 04-06-2010
Code:
awk 'NR==FNR{arr[$0];next}$0 in arr{next}1' File_A File_B

You can change the input file order to get the opposite.
# 4  
Old 04-06-2010
Bug Shell Script: Compare 2 Files and Extract Unique Elements of Each File into 2 New Files

You guys are awesome! Masters of the one-liners! Works perfectly! Thanks so much for your help. Smilie
Matt
# 5  
Old 04-06-2010
Quote:
Originally Posted by mlv_99
Let me re-phrase the question again.
Now is clear Smilie

Quote:
Originally Posted by mlv_99
Resultant File_New_A will contain only unique elements found only in File_A when compared with File_B.
Code:
grep -vf File_B File_A > File_New_A

Quote:
Originally Posted by mlv_99
File_New_B will contain only unique elements found in only File_B when compared with File_A.
Code:
grep -vf File_A File_B > File_New_B

# 6  
Old 04-06-2010
Shell Script: Compare 2 Files and Extract Unique Elements of Each File into 2 New Files

Smilie Thanks Danmero! Exactly what I was looking for!
-Matt
# 7  
Old 04-06-2010
Hi,

I have a question. My post was not published, I have no idea why. But, it is a good.Smilie
The command "comm" is not suitable for this? If not, why?

Greeting
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to get script to create a new file that lists folder content sorted by size?

I have a script that sorts and processes unsorted files to newly created directories. Its working great, but I am trying to understand the leanest method to get the script to create an additional file within each newly created directory that: Contains a list of all files in the directory... (4 Replies)
Discussion started by: Braveheart
4 Replies

2. Shell Programming and Scripting

• Write a shell script that upon invocation shows the time and date and lists all the logged-in user

help me (1 Reply)
Discussion started by: sonu pandey
1 Replies

3. Shell Programming and Scripting

Removing duplicate entries from edge-lists

I have a file which has connections given as: A B 0.1 B C 5.8 C B 5.8 E F 0.67 B A 0.1 A B and B A are same, so I want to remove one of them. Same with BC and CB. Desired output: A B 0.1 B C 5.8 E F 0.67 (2 Replies)
Discussion started by: Sanchari
2 Replies

4. Shell Programming and Scripting

get the lists

I expert, I may cross post something similar but I dirtyed my quesion somehow to be clear in the thread #cat file1 88dee gcc: Grok for callconvention-hard to enable hard float a2ad2 eglibc: package mtrace separately 61487 python: bump PR of packages after update of distutils.bbclass... (1 Reply)
Discussion started by: yanglei_fage
1 Replies

5. Shell Programming and Scripting

Shell script that lists files with different owner than the folder

Hello, I'm trying to write a script which is listing files based on different preferences, like filetype or permissions. All is fine, except for one: I want to list files in /home which has a different owner than the home directory it is in. Here is an example: /home/UserA is the directory, and... (10 Replies)
Discussion started by: Zwiebi
10 Replies

6. Shell Programming and Scripting

Bash script to compare two lists

Hi, I do little bash scripting so sorry for my ignorance. How do I compare if the two variable not match and if they do not match run a command. I was thinking a for loop but then I need another for loop for the 2nd list and I do not think that would work as in the real world there could... (2 Replies)
Discussion started by: GermanJulian
2 Replies

7. Shell Programming and Scripting

C shell: Working with lists

Hello Unix Gurus, I have: A list of parameters that repeat (in .txt file) Example: params.txt Series: XYZ Manufacturer: ... Software Version: ... Year made: ... Series Series: XYZ Manufacturer: ... Software Version: ... Year made: ... Series Series: ABC Manufacturer: ... ... (7 Replies)
Discussion started by: lapiduslost
7 Replies

8. Shell Programming and Scripting

Editing lists of integers in 1d files with bash shell

Hi, I need a script that will: 1. Go through about 20 different folders, each containing about 20 1d files. The 1d files go something like this: 22.253 37.707 78.117 112.374 127.944 156.067 180.956 233.785 249.256 ... (1 Reply)
Discussion started by: ac130pilot
1 Replies

9. Shell Programming and Scripting

Help with a shell script to concatenate lists together

Below is a description of what im trying to achieve: Write a shell script to concatenate lists together, and output the resulting list. Do not include any argument that is a sub-list of the entire list. (The script will clean the list of any redundant items.) You must preserve the original order... (1 Reply)
Discussion started by: rfourn
1 Replies

10. Shell Programming and Scripting

Check lists for Unix Shell Programming

Hi all, Can anyone provide me any checklists or a list of steps I should follow before executing my scripts. Could also tell me if there are any other standards to be followed while shell programming like naming conventions for variables etc. Your help would be much appreciated. Regards,... (2 Replies)
Discussion started by: srikanth_ksv
2 Replies
Login or Register to Ask a Question