How can i prepare a file by comparing two other files?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How can i prepare a file by comparing two other files?
# 1  
Old 02-15-2008
Shell script for merging two files without duplicating the data

File 1 data:

TestA
TestB
TestC


File 2 data:

TestA
TestD
TestE


My Output File (pick all from both and create a file without duplicate data)
----------------------------------------------------------------------

TestA
TestB
TestC
TestD
TestE

----------------------------------------------------------------
here, TestA comes in both files, i dont want to duplicate it (appear multiple times) in my output file. How can i do it?

Thanks in advance,

-mac

Last edited by manmohanpv; 02-16-2008 at 04:46 AM..
# 2  
Old 02-16-2008
Code:
awk '!x[$0]++' file1 file2

# 3  
Old 02-16-2008
I've misunderstood the request. The previous script removes duplicate entries but doesn't sort the data.

You can simply use:

Code:
sort -u file1 file2

# 4  
Old 02-18-2008
sort+sed+comm

Hi,

Follow one should be ok for you.

Code:
sort file1 > t1
sort file2 > t2
comm t1 t2 | sed 's/	//g'
rm t1 t2

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to prepare a CSV table with inputs from multiple files

Hi, We have thousands of Video files and we need to prepare a table with few of it's parameters/values from their Mediainfo output. Here I have 3 sample mediainfo output text files named Vid1, Vid2 & Vid3 so on... (such we have thousands) and each file has exactly 3 lines. $ ls... (6 Replies)
Discussion started by: prvnrk
6 Replies

2. Shell Programming and Scripting

How can i prepare grant staement with 2 files ?

---file1 ( tables A B C D E F ... ... Z ---file2 Joe Bob Mary Sally Fred Elmer David (1 Reply)
Discussion started by: rocking77
1 Replies

3. Shell Programming and Scripting

Prepare file run report

I have a requirement to prepare a report. We validate some incoming data fields and create validation_error reports which will contain records which do not pass validation. Once files are processed they will all be dropped under one folder. EMPLOYEE_20140915.txt... (8 Replies)
Discussion started by: member2014
8 Replies

4. Shell Programming and Scripting

Comparing two files and creating a new file

Hi, I want to compare two files based on the data in their first column. Both the files are not equal in their number of columns and number of entries or rows. The entries (only the first column) of the file1 should be compared with the entries (only the first column) of the file2. If the... (3 Replies)
Discussion started by: begin_shell
3 Replies

5. Shell Programming and Scripting

comparing 2 files and creating third file with uncommon content

I want to compare 2 files and create third file with uncommon content. e.g. file1 ajay suhas tom nisha vijay mahish file2 ajay suhas tom nisha expected output file content vijay mahish Is it possible in single command ? Thanks, Ajay (6 Replies)
Discussion started by: ajaypatil_am
6 Replies

6. UNIX for Dummies Questions & Answers

Comparing a batch of files to a test file

Hi I am writing a script to run a loop through a directory and run a diff on each file against my test file. I then want to time how long each file to process (not sure how time works), as well as how long all the files took as a whole to process. Here is my code #!/bin/bash #Old... (1 Reply)
Discussion started by: ladyAnne
1 Replies

7. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

8. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies

9. Shell Programming and Scripting

comparing files to contents of a file

Hi I have a problem trying to run a while statement. I have files under one directory that i need to compare to a value in filex and update that file with the result files in the directory are DFC1. DFC5. DFC345. DFC344. DFC9. The program i am trying to run will take the number... (3 Replies)
Discussion started by: SummitElse
3 Replies

10. Shell Programming and Scripting

shell script comparing files in a file

There is a text file that contains the data in the following format: COLUMN1 COLUMN2 ABC 1 ABC 2 ABC 3 DEF 4 DEF 5 XYZ 7 We have to create a second text file... (4 Replies)
Discussion started by: raina_nalin
4 Replies
Login or Register to Ask a Question