Sponsored Content
Top Forums Shell Programming and Scripting How to sort and compare files in more efficient manner? Post 302810579 by Vikram_Tanwar12 on Wednesday 22nd of May 2013 06:25:14 AM
Old 05-22-2013
How to sort and compare files in more efficient manner?

Hello All,

Iam using below method to sort and compare files. First iam doing sorting and changing the same file and then doing comparing and taking the final result to another file.

Code:
sort -o temp.txt file1
mv temp.txt file1
sort -o temp.txt file2
mv temp.txt file2
sort -o temp.txt file3
mv temp.txt file3
comm -23 file1 file2 > file4
comm -23 file4 file3 > file5
awk -v RS='' -v OFS="," '$1=$1'  file5 > file6
mv file6  file5

Is their a more better or clear way of doing the above task.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Is there an efficient way in finding and deleting files?

Hi I have a script to find and delete the files which are say, noDaysOld, I am interested to find the number of such files I am fniding for deleting and then deleting it. So, the script I wrote, first finds the number of such files and then deletes, clearly this is two different steps. ... (3 Replies)
Discussion started by: guruparan18
3 Replies

2. Shell Programming and Scripting

Why do we use sort before compare ?

Dear All. Im trying to know how exactly the command "compare" works, does it compare line by line or field by field, and the most important thing is that why the files have to be sorted before we compare them? Thanks in advance (7 Replies)
Discussion started by: yahyaaa
7 Replies

3. UNIX for Dummies Questions & Answers

sort and compare files

Hi, I have around 14-15 files and i want to sort all the files and then compare. I dont want to sort them and store in a different file and then compare. I want to compare two files at a time. Is there a way we can do this using a single command? (5 Replies)
Discussion started by: dnat
5 Replies

4. Shell Programming and Scripting

HELP: I need to sort a text file in an uncommon manner, can't get desired results

Hi All I have a flat text file. Each line in it contains a "/full path/filename". The last three columns are predictable, but directory depth of each line varies. I want to sort on the last three columns, starting from the last, 2nd last and 3rd last. In that order. The last three columns... (6 Replies)
Discussion started by: JakeKatz
6 Replies

5. Emergency UNIX and Linux Support

Appending the contents of two files in computational manner

Hi, I have two files say file 1 file 2 File1 1 2 4 5 File 2 asdf adf How to get the ouput something like asdf1 adf1 asdf2 adf2 asdf4 adf4 asdf5 (5 Replies)
Discussion started by: ecearund
5 Replies

6. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

7. Programming

Help with make this Fortran code more efficient (in HPC manner)

Hi there, I had run into some fortran code to modify. Obviously, it was written without thinking of high performance computing and not parallelized... Now I would like to make the code "on track" and parallel. After a whole afternoon thinking, I still cannot find where to start. Can any one... (3 Replies)
Discussion started by: P_E_M_Lee
3 Replies

8. Shell Programming and Scripting

comm command -- Sort and compare two files

Team, I have two files and I am trying to find the lines unique to file1. So i have executed the below command at shell prompt and got the correct results comm -23 <(sort test) <(sort test1) When i run the same command in Bash shell script, i got the correct results. But when i run... (5 Replies)
Discussion started by: forums123456
5 Replies

9. Shell Programming and Scripting

Most efficient method to extract values from text files

I have a list of files defined in a single file , one on each line.(No.of files may wary each time) eg. content of ETL_LOOKUP.dat /data/project/randomname /data/project/ramname /data/project/raname /data/project/radomname /data/project/raame /data/project/andomname size of these... (5 Replies)
Discussion started by: h0x0r21
5 Replies

10. Shell Programming and Scripting

More efficient for loop on files

is there a more efficient way to write the following code: for eachlfile in $(ls -ltcrd ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}* 2>/dev/null | awk '{print $NF}') do echo "==========================" ... (4 Replies)
Discussion started by: SkySmart
4 Replies
diff3(1)						      General Commands Manual							  diff3(1)

Name
       diff3 - 3-way differential file comparison

Syntax
       diff3 [-ex3] file1 file2 file3

Description
       The command compares three versions of a file, and publishes the ranges of text that disagree, flagged with the following codes:

	  ====	      all three files differ

	  ====1       file1 is different

	  ====2       file2 is different

	  ====3       file3 is different

       The type of change needed to convert a given range of a given file to some other is indicated in one of these ways:

	  f : n1 a    Text is to be appended after line number n1 in file f, where f = 1, 2, or 3.

	  f : n1 , n2 c
		      Text is to be changed in the range line n1 to line n2.  If n1 = n2, the range may be abbreviated to n1.

       The original contents of the range follows immediately after a c indication.  When the contents of two files are identical, the contents of
       the lower-numbered file is suppressed.

Options
       -3   Produces an editor script containing the changes between file1 and file2 that are to be incorporated into file3.

       -e	   Produces an editor script containing the changes between file2 and file3 that are to be incorporated into file1.

       -x	   Produces an editor script containing the changes among all three files.

Examples
       Under the -e option, publishes a script for the editor that incorporates into file1 all changes between file2 and  file3  -  that  is,  the
       changes	that would normally be flagged ==== and ====3.	Option -x (-3) produces a script to incorporate only changes flagged ==== (====3).
       The following command applies the resulting script to `file1':
       (cat script; echo '1,$p') | ed - file1

Restrictions
       Text lines that consist of a single `.'	defeat -e.

Files
       /tmp/d3?????
       /usr/lib/diff3

See Also
       cmp(1), comm(1), diff(1), dffmk(1), join(1), sccsdiff(1), uniq(1)

																	  diff3(1)
All times are GMT -4. The time now is 10:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy