12-21-2009
Quote:
Originally Posted by
gaurav1086
Hello brett,
there are several known algorithms for that purpose. I feel the K-way-merge algorithm is the best one that I have used for that purpose.
In addition to that you can put up a condition to remove the duplicate entries
Or otherwise you can first remove the duplicate entries from the two sources after/before sorting and then implement the K-way merge algorithm .
Feel free to reply if any doubts.
You would like to check this in case
K-Way Merge | Get K-Way Merge at SourceForge.net
Regards.
Is the key way merge algorithm for already sorted files??
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all,
please give me the commands using which i can compare 2 sorted files and get the difference in third file, indiating where the difference is from either file1 or file2.
as:
File1 (Original file)
GARRY
JOHN
JULIE
SAM
---------------
File2
DEV
GARRY
JOHN
JOHNIEE (7 Replies)
Discussion started by: varungupta
7 Replies
2. Shell Programming and Scripting
Say i have 2 files in the giving format:
file1
1 2 3 4
1 2 3 4
1 2 3 4
file2
1 2 3 4
1 2 3 4
1 2 3 4
I have a PERL code (loaned by one of u -i forgot who - thanks!) that extracts the 2nd column from each file and append horizontally to a new file:
perl -ane 'push @{$L->}, $F; close... (1 Reply)
Discussion started by: epi8
1 Replies
3. Shell Programming and Scripting
I am a beginner in script writing, i tried to do the following
I have a set of files sorted by date in the format YYMMDD.s and .x and .r
I need to concatenate a header file to these sets of files so I used the following code
echo "enter Swath number"
read s
echo "please enter first date and... (2 Replies)
Discussion started by: docaia
2 Replies
4. Shell Programming and Scripting
Hi, I need a script that will:
1. Go through about 20 different folders, each containing about 20 1d files. The 1d files go something like this:
22.253 37.707 78.117 112.374 127.944 156.067 180.956 233.785 249.256 ... (1 Reply)
Discussion started by: ac130pilot
1 Replies
5. UNIX for Dummies Questions & Answers
the sorting is based on name of file,
file size
modification time stamps o f file
it should dislay the output in the following format
"." and ".." enteries should be ignored
please give some idea how to do it (1 Reply)
Discussion started by: pappu kumar jha
1 Replies
6. Shell Programming and Scripting
I have multiple files; each file contains a certain data in a column view
simply i want to combine all those files into one file in columns
example
file1:
a
b
c
d
file 2:
1
2
3
4
file 3:
G (4 Replies)
Discussion started by: ahmedamro
4 Replies
7. Shell Programming and Scripting
Hi
I have a requirement like below
I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory
My files looks like this
PGABOLTXML1D_201108121235.xml... (1 Reply)
Discussion started by: saidutta123
1 Replies
8. Shell Programming and Scripting
Hi,
I am working in bash in Mac OSX, I have following 'input.txt' file:
<INFO> HypoTestTool: >>> Done running HypoTestInverter on the workspace combined
<INFO> HypoTestTool: The computed upper limit is: 11 +/- 1.02651
<INFO> HypoTestTool: expected limit (median) 11
<INFO> HypoTestTool: ... (13 Replies)
Discussion started by: Asif Siddique
13 Replies
9. Shell Programming and Scripting
Hello,
I want to merge multiple files (under hundreds folders) side by side. File name are the same but folder are different.
like
folder1/same_name.txt
folder2/same_name.txt
folder3/same_name.txt
......Normally it can be done as
paste /different_path*/same_name.txt > merged_file.txtbut... (2 Replies)
Discussion started by: yifangt
2 Replies
10. Shell Programming and Scripting
Can anyone please help me i have 2 text files setup like the one below.
Textfile1:
randomemail1:randompassword1
randomemail2:randompassword2
randomemail3:randompassword3
randomemail4:randompassword4
randomemail5:randompassword5
Textfile2:
randompassword1:randomphrase1... (8 Replies)
Discussion started by: nufc
8 Replies
LEARN ABOUT DEBIAN
unsort
UNSORT(1) BSD General Commands Manual UNSORT(1)
NAME
unsort -- reorder lines in a file in semirandom ways
SYNOPSIS
unsort [-hvrpncmMsz0l] [--help] [--version] [--random] [--heuristic] [--identity] [--concatenate] [--merge] [--merge-random] [--seed integer]
[--zero-terminated] [--null] [--linefeed] [file ...]
DESCRIPTION
unsort prints the lines in the input files (or standard input) in semi-random order. Available algorithms are a Mersenne Twister based PRNG
and a heuristic algorithm that aims to create a subjective even distribution.
Command line options
-h, --help
Display a concise summary of the available options and argument syntax.
-v, --version
Display version and copyright information.
-r, --random
Use the Mersenne Twister based randomization algorithm.
-p, --heuristic
Use the heuristic "shuffling" algorithm which permutes the lines in such a way that they're spread more or less evenly in the output.
This is the default.
-n, --identity
Do not reorder lines in the input. Useful if you just want to merge the files.
-r, --concatenate
Concatenate all input files then apply the shuffling algorithm to the result as a whole.
-m, --merge
Shuffle all input files seperately then merge the result. Equal-sized files will be merged in the order in which they appear on the
command line.
-M, --merge-random
Shuffle all input files seperately then merge the result. Equal-sized files will be merged in random order. This is the default.
-s, --seed integer
Use this integer as a seed, instead of random data from the environment.
-z, --zero-terminated, -0, --null
Lines are terminated with a character.
-l, --linefeed
Lines are terminated with a
character. This is the default.
SEE ALSO
sort(1)
Free Software June 1, 2019 Free Software