Sponsored Content
Top Forums Shell Programming and Scripting Search Pattern and combine into single file Post 302825709 by balajesuri on Tuesday 25th of June 2013 02:48:46 AM
Old 06-25-2013
Quote:
Originally Posted by navkanwal
2 files and want the output file: {No for using FOR loop because I got 22 million lines}
Tried that "It processes only 8000 records per hour"
I need a faster way out !!!
Quote:
Originally Posted by navkanwal
I tried the following command, but it only gives me matches pattern:
awk -F";" 'NR==FNR{A[$1]=$0;next}$1 in A{$0=$0","A[$1];print}' fileA fileB
If you load 22 millions lines in an array, it will definitely lead to performance issues. And here's your awk one-liner:
Code:
awk -F";" 'NR==FNR{A[$1]=$2;next} {if (A[$1]){$0=$0";"$1";"A[$1];print}else{print}}' fileB fileA

And, with the sample fileA and fileB you provided in post #1, using join I got the output which I've pasted in post #2. Not sure, why you're getting a different output. Could you please re-check.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search file for pattern and grab some lines before pattern

I want to search a file for a string and then if the string is found I need the line that the string is on - but also the previous two lines from the file (that the pattern will not be found in) This is on solaris Can you help? (2 Replies)
Discussion started by: frustrated1
2 Replies

2. Shell Programming and Scripting

Search a pattern in a file with contents in a single line

Hi all I am searching for a pattern in a file . The file content is in a single line.If am doing a grep or sed for the a particular pattern am getting whole file. I want the result in different lines. attaching the file for reference search pattern "/xxxxxx/hhhh/tttttttt/sss/" and... (4 Replies)
Discussion started by: sparks
4 Replies

3. Shell Programming and Scripting

awk pattern match and search in single statement

Hi All, I am trying to alter all lines between EXEC SQL and END-EXEC that have an INCLUDE in them. The following code search="INCLUDE " cp -f ${WORK}/$file.in ${WORK}/$file.wrk2 for item in `echo $search `; do > ${WORK}/$file.wrk1 awk -vITEM="$item" '{ if ( $0... (3 Replies)
Discussion started by: Bruble
3 Replies

4. Shell Programming and Scripting

Combine in single file

I have 2 files: phone.txt and mobile.txt ex. phone.txt MOBILENO|DISABLE_DATE 919687877754|9/1/2011| 919687877762|9/1/2011| 919687880573|9/1/2011| 919687882658|9/2/2011| Ex. mobile.txt MOBILENO |TIME 919687880573|2011-09-17 12:23:40| 919687882658|2011-10-10 21:15:33|... (4 Replies)
Discussion started by: khingx
4 Replies

5. Shell Programming and Scripting

how to find first files in a directory and combine them as a single file?

i have below list of files in a directory. /root/admin/files/file1.txt /root/admin/files/file2.txt /root/admin/files/file3.txt /root/admin/files/pattern.txt /root/admin/files/server.txt i need combine the above text files in the below sequence, file1.txt, pattern.txt,server.txt =>... (8 Replies)
Discussion started by: vel4ever
8 Replies

6. Shell Programming and Scripting

Combine Multiple Files into Single One File One after other

I am trying to combine 4 .dat files into one single Output file Inputs are:- file123.dat, file256.dat, file378.dat & file490 Expected Output:- FileName=file1 {text from file1} EOF {blank line} FileName=file2 {text from file2} EOF {blank line} FileName=file3 {text from file3} EOF... (4 Replies)
Discussion started by: lancesunny
4 Replies

7. Shell Programming and Scripting

Search for a pattern in a String file and count the occurance of each pattern

I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported Input file is a free flowing file without any format example of output ERR-00001=5 .... ERR-01010=10 ..... ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies

8. Shell Programming and Scripting

How to use sed to search a particular pattern in a file backward after a pattern is matched.?

Hi, I have two files file1.txt and file2.txt. Please see the attachments. In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies

9. Shell Programming and Scripting

Search Pattern and Print lines in Single Column

Hi Experts I have small query where I request the into a single file Suppose: File1: {Unique entries} AA BB CC DD FileB: AA, 123 AA, 234 AA, 2345 CC, 123 CC, 5678 DD,123 BB, 7890 (5 Replies)
Discussion started by: navkanwal
5 Replies
COMBINE(1)																COMBINE(1)

NAME
combine - combine sets of lines from two files using boolean operations SYNOPSIS
combine file1 and file2 combine file1 not file2 combine file1 or file2 combine file1 xor file2 _ file1 and file2 _ _ file1 not file2 _ _ file1 or file2 _ _ file1 xor file2 _ DESCRIPTION
combine combines the lines in two files. Depending on the boolean operation specified, the contents will be combined in different ways: and Outputs lines that are in file1 if they are also present in file2. not Outputs lines that are in file1 but not in file2. or Outputs lines that are in file1 or file2. xor Outputs lines that are in either file1 or file2, but not in both files. "-" can be specified for either file to read stdin for that file. The input files need not be sorted, and the lines are output in the order they occur in file1 (followed by the order they occur in file2 for the two "or" operations). Bear in mind that this means that the operations are not commutative; "a and b" will not necessarily be the same as "b and a". To obtain commutative behavior sort and uniq the result. Note that this program can be installed as "_" to allow for the syntactic sugar shown in the latter half of the synopsis (similar to the test/[ command). It is not currently installed as "_" by default, but you can alias it to that if you like. SEE ALSO
join(1) AUTHOR
Copyright 2006 by Joey Hess <joey@kitenet.net> Licensed under the GNU GPL. moreutils 2012-04-09 COMBINE(1)
All times are GMT -4. The time now is 01:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy