Sponsored Content
Top Forums Shell Programming and Scripting Best performance to merge two files Post 303027330 by green_k on Thursday 13th of December 2018 11:57:46 AM
Old 12-13-2018
Best performance to merge two files

Hi Gurus,


I need to merge two files.


file1 (small file, only one line)
Code:
this is first line

file2 (large file)
Code:
 abc
def
ghi
...

I use below command to merge the file, since the file2 is really large file, the command read whole file2, the performance is not good.
Code:
cat file1 > file3
cat file2 >> file2

I searched internet, there is suggestion to use
Code:
sed -i '1 i\anything' file or 
sed -i '1i text' filename

the command thrown out error
Code:
sed: illegal option -- i

please provide your valuable suggestion.

my os is SunOS 5.10 Generic_150400-04 sun4v sparc sun4v


thanks in advance

Last edited by green_k; 12-13-2018 at 01:05 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

help in merge files

I have created these files in a particular directory a_1.txt a_2.txt ... .. a_n.txt Each file has a single line. I want to write a output file a.txt, which will have concantated value of string from all the files. which utility should i use. copy, cat or paste???? Can anyone help... (5 Replies)
Discussion started by: u263066
5 Replies

2. Shell Programming and Scripting

How to merge files

Hello guys, I gotta question, i have a lot of log files (simple text) and i need to merge them in group of 10 files, one next to the other, that have sense? For example, i have the files: File1 File2 File3 File4 . . File100 I need to merge the contents of each file into a new file... (3 Replies)
Discussion started by: lestat_ecuador
3 Replies

3. Shell Programming and Scripting

merge files

Hi, i have the files f1 and f2 like: files f1: c1 a1 c2 a2 c3 a3 file f2: c1 b1 c2 b2 c3 b3 i want merge the f1 and f2 file to f3 file like: c1 a1 b1 c2 a2 b3 c3 a3 b3........ .... . . please help me onthis..... (5 Replies)
Discussion started by: koti_rama
5 Replies

4. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

5. Shell Programming and Scripting

merge files

hi i have two files file1 1234567 1234678 1234679 file2 98765|jjkskk|9393|iyutr 98765|kkooo|9393|hjjjd 98765|abcvfg|9393|sskds output should be 1234567|jjkskk|9393|iyutr 1234678|kkooo|9393|hjjjd 1234679|abcvfg|9393|sskds (5 Replies)
Discussion started by: mad_man12
5 Replies

6. Shell Programming and Scripting

How can i merge these two files into several...

Given are File A and File B File A has for example 5 lines: AAA BBB CCC DDD EEE File B has 3 lines: 111 222 333 How can i merge A and B into: 111 222 333 AAA (first line from A) then a new file: (4 Replies)
Discussion started by: Y-T
4 Replies

7. UNIX for Dummies Questions & Answers

Merge files

Hi, I would like to know how can I merge files based on their coordinates, but mantaining the score of each file in the output file like: Note: 1st column is for chromosome, 2nd for start, 3rd for end of segment, 4th for score file1: 1 200 300 20 1 400 500 30 file2: 1 200 350 30 1... (1 Reply)
Discussion started by: fadista
1 Replies

8. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

9. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies

10. Programming

DB2 - Performance Issue using MERGE option

Dear Team, I am using DB2 v10.5 and trying to load huge data using MERGE option ( 10-12 Million) using below query. Basically it loads data from staging to target . Staging table schemaname.Customer_Staging has 12 Million records . Runs for 25 Mins for just select query. But while doing merge... (2 Replies)
Discussion started by: Perlbaby
2 Replies
comm(1) 							   User Commands							   comm(1)

NAME
comm - select or reject lines common to two files SYNOPSIS
comm [-123] file1 file2 DESCRIPTION
The comm utility reads file1 and file2, which must be ordered in the current collating sequence, and produces three text columns as output: lines only in file1; lines only in file2; and lines in both files. If the input files were ordered according to the collating sequence of the current locale, the lines written will be in the collating sequence of the original lines. If not, the results are unspecified. OPTIONS
The following options are supported: -1 Suppresses the output column of lines unique to file1. -2 Suppresses the output column of lines unique to file2. -3 Suppresses the output column of lines duplicated in file1 and file2. OPERANDS
The following operands are supported: file1 A path name of the first file to be compared. If file1 is -, the standard input is used. file2 A path name of the second file to be compared. If file2 is -, the standard input is used. USAGE
See largefile(5) for the description of the behavior of comm when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). EXAMPLES
Example 1 Printing a list of utilities specified by files If file1, file2, and file3 each contain a sorted list of utilities, the command example% comm -23 file1 file2 | comm -23 - file3 prints a list of utilities in file1 not specified by either of the other files. The entry: example% comm -12 file1 file2 | comm -12 - file3 prints a list of utilities specified by all three files. And the entry: example% comm -12 file2 file3 | comm -23 -file1 prints a list of utilities specified by both file2 and file3, but not specified in file1. ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of comm: LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 All input files were successfully output as specified. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cmp(1), diff(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5) SunOS 5.11 3 Mar 2004 comm(1)
All times are GMT -4. The time now is 05:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy