Sponsored Content
Top Forums Shell Programming and Scripting Search Pattern and combine into single file Post 302825709 by balajesuri on Tuesday 25th of June 2013 02:48:46 AM
Old 06-25-2013
Quote:
Originally Posted by navkanwal
2 files and want the output file: {No for using FOR loop because I got 22 million lines}
Tried that "It processes only 8000 records per hour"
I need a faster way out !!!
Quote:
Originally Posted by navkanwal
I tried the following command, but it only gives me matches pattern:
awk -F";" 'NR==FNR{A[$1]=$0;next}$1 in A{$0=$0","A[$1];print}' fileA fileB
If you load 22 millions lines in an array, it will definitely lead to performance issues. And here's your awk one-liner:
Code:
awk -F";" 'NR==FNR{A[$1]=$2;next} {if (A[$1]){$0=$0";"$1";"A[$1];print}else{print}}' fileB fileA

And, with the sample fileA and fileB you provided in post #1, using join I got the output which I've pasted in post #2. Not sure, why you're getting a different output. Could you please re-check.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search file for pattern and grab some lines before pattern

I want to search a file for a string and then if the string is found I need the line that the string is on - but also the previous two lines from the file (that the pattern will not be found in) This is on solaris Can you help? (2 Replies)
Discussion started by: frustrated1
2 Replies

2. Shell Programming and Scripting

Search a pattern in a file with contents in a single line

Hi all I am searching for a pattern in a file . The file content is in a single line.If am doing a grep or sed for the a particular pattern am getting whole file. I want the result in different lines. attaching the file for reference search pattern "/xxxxxx/hhhh/tttttttt/sss/" and... (4 Replies)
Discussion started by: sparks
4 Replies

3. Shell Programming and Scripting

awk pattern match and search in single statement

Hi All, I am trying to alter all lines between EXEC SQL and END-EXEC that have an INCLUDE in them. The following code search="INCLUDE " cp -f ${WORK}/$file.in ${WORK}/$file.wrk2 for item in `echo $search `; do > ${WORK}/$file.wrk1 awk -vITEM="$item" '{ if ( $0... (3 Replies)
Discussion started by: Bruble
3 Replies

4. Shell Programming and Scripting

Combine in single file

I have 2 files: phone.txt and mobile.txt ex. phone.txt MOBILENO|DISABLE_DATE 919687877754|9/1/2011| 919687877762|9/1/2011| 919687880573|9/1/2011| 919687882658|9/2/2011| Ex. mobile.txt MOBILENO |TIME 919687880573|2011-09-17 12:23:40| 919687882658|2011-10-10 21:15:33|... (4 Replies)
Discussion started by: khingx
4 Replies

5. Shell Programming and Scripting

how to find first files in a directory and combine them as a single file?

i have below list of files in a directory. /root/admin/files/file1.txt /root/admin/files/file2.txt /root/admin/files/file3.txt /root/admin/files/pattern.txt /root/admin/files/server.txt i need combine the above text files in the below sequence, file1.txt, pattern.txt,server.txt =>... (8 Replies)
Discussion started by: vel4ever
8 Replies

6. Shell Programming and Scripting

Combine Multiple Files into Single One File One after other

I am trying to combine 4 .dat files into one single Output file Inputs are:- file123.dat, file256.dat, file378.dat & file490 Expected Output:- FileName=file1 {text from file1} EOF {blank line} FileName=file2 {text from file2} EOF {blank line} FileName=file3 {text from file3} EOF... (4 Replies)
Discussion started by: lancesunny
4 Replies

7. Shell Programming and Scripting

Search for a pattern in a String file and count the occurance of each pattern

I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported Input file is a free flowing file without any format example of output ERR-00001=5 .... ERR-01010=10 ..... ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies

8. Shell Programming and Scripting

How to use sed to search a particular pattern in a file backward after a pattern is matched.?

Hi, I have two files file1.txt and file2.txt. Please see the attachments. In file2.txt (which actually is a diff output between two versions of file1.txt.), I extract the pattern corresponding to 1172c1172. Now ,In file1.txt I have to search for this pattern 1172c1172 and if found, I have to... (9 Replies)
Discussion started by: saurabh kumar
9 Replies

9. Shell Programming and Scripting

Search Pattern and Print lines in Single Column

Hi Experts I have small query where I request the into a single file Suppose: File1: {Unique entries} AA BB CC DD FileB: AA, 123 AA, 234 AA, 2345 CC, 123 CC, 5678 DD,123 BB, 7890 (5 Replies)
Discussion started by: navkanwal
5 Replies
JOIN(1) 						      General Commands Manual							   JOIN(1)

NAME
join - relational database operator SYNOPSIS
join [ options ] file1 file2 DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard input is used. File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in each line. There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con- sists of the common field, then the rest of the line from file1, then the rest of the line from file2. Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis- carded. These options are recognized: -an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2. -e s Replace empty output fields by string s. -jn m Join on the mth field of file n. If n is missing, use the mth field in each file. -o list Each output line comprises the fields specified in list, each element of which has the form n.m, where n is a file number and m is a field number. -tc Use character c as a separator (tab character). Every appearance of c in a line is significant. SEE ALSO
sort(1), comm(1), awk(1) BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort. The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous. 7th Edition April 29, 1985 JOIN(1)
All times are GMT -4. The time now is 01:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy