Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Removing file lines that each match to a different patterns
Top Forums Shell Programming and Scripting Removing file lines that each match to a different patterns Post 302412984 by jim mcnamara on Wednesday 14th of April 2010 07:11:57 AM
Old 04-14-2010
If your second example 1:2:3 is representative of the actual file contents of the small file, i.e., it has no prefix and no suffixed data either
Code:
awk -F':'  ' FILENAME=="smallfile" {arr[$1 $2 $3]++}
               FILENAME=="bigfile" {tmp=$2 $3 $4; if(tmp in arr) {next}; print $0 }
           ' smallfile bigfile >  newfile

Also
Code:
export LC_ALL=C

may help your error message language problem.
Last edited by jim mcnamara; 04-14-2010 at 08:37 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed/awk help to match list of patterns and remove from org file

Hi, From the pattern mentioned below remove lines based on pattern range. Conditions 1 Look For all lines starting with ALTER TABLE and Ending with ; and contains the word MOVE.I wanto to remove these lines from the file sample below. Note : The above pattern list could be found in... (1 Reply)
Discussion started by: rajan_san
1 Replies

2. Shell Programming and Scripting

Searching patterns in 1 file and deleting all lines with those patterns in 2nd file

Hi Gurus, I have a file say for ex. file1 which has 3500 lines in it which are different account numbers and another file (file2) which has 230000 lines in it. I want to read all the lines in file1 and delete all those lines from file2 which has that same pattern as in file1. I am not quite... (4 Replies)
Discussion started by: toms
4 Replies

3. Shell Programming and Scripting

print lines which match multiple patterns

Hi, I have a text file as follows: 11:38:11.054 run1_rdseq avg_2-5 999988.0000 1024.0000 11:50:52.053 run3_rdrand 999988.0000 1135.0 128.0417 11:53:18.050 run4_wrrand avg_2-5 999988.0000 8180.5833 11:55:42.051 run4_wrrand avg_2-5 999988.0000 213.8333 11:55:06.053... (2 Replies)
Discussion started by: annazpereira
2 Replies

4. Shell Programming and Scripting

Match multiple patterns in a file and then print their respective next line

Dear all, I need to search multiple patterns and then I need to print their respective next lines. For an example, in the below table, I will look for 3 different patterns : 1) # ATC_Codes: 2) # Generic_Name: 3) # Drug_Target_1_Gene_Name: #BEGIN_DRUGCARD DB00001 # AHFS_Codes:... (3 Replies)
Discussion started by: AshwaniSharma09
3 Replies

5. Shell Programming and Scripting

Retrieve lines that match any occurence in a list of patterns

I have two files. The first containing a header and six columns of data. Example file 1: Number SNP ID dbSNP RS ID Chromosome Result_Call Physical Position 787066 SNP_A-8575395 RS6650104 1 NOCALL 564477 786872 SNP_A-8575125 RS10458597 1 AA ... (13 Replies)
Discussion started by: Selftaught
13 Replies

6. Shell Programming and Scripting

Match 2 different patterns and print the lines

Hi, i have been trying to extract multiple lines based on two different patterns as below:- file1 @jkm|kdo|aas012|192.2.3.1 blablbalablablkabblablabla sjfdsakfjladfjefhaghfagfkafagkjsghfalhfk fhajkhfadjkhfalhflaffajkgfajkghfajkhgfkf jahfjkhflkhalfdhfwearhahfl @jkm|sdf|wud08q|168.2.1.3... (8 Replies)
Discussion started by: redse171
8 Replies

7. UNIX for Dummies Questions & Answers

Match patterns from another file and tag

Hi all, I have a file , which has 6 tab delimited fields, with $3 and $4 subfielded with spaces. I wamt to match cols $2,$3,$4 of tmp1 with tmp2, ..and then flag the 5th col if found. tmp1 1756 Xerm XermA XermB XermC XermD AA TT AA GG A 1 1763 Xerm XermA XermB XermC... (3 Replies)
Discussion started by: senhia83
3 Replies

8. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

9. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies

10. Shell Programming and Scripting

Selecting text on multiple lines, then removing a beginning and end patterns

I have a file similar to the below. I am selecting only the paragraphs with @inlineifset. I am using the following command sed '/@inlineifset/,/^ *$/!d; s/@inlineifset{mrg, @btpar{@//' $flnm >> $ofln This produces @section Correlations between seismograms,,,,}} ... (5 Replies)
Discussion started by: Danette
5 Replies

LEARN ABOUT DEBIAN

oakdecode

oakdecode(1)						      General Commands Manual						      oakdecode(1)

NAME

       oakdecode - Decode an OAKT printer stream into human readable form.

SYNOPSIS

       oakdecode [options] <OAKT-file

DESCRIPTION

       oakdecode decodes an OAKT printer stream into human readable form.

       An OAKT printer stream is the printer langauge used by the HP Color LaserJet 1500 and other printers.

COMMAND LINE OPTIONS

       These are the options that can appear on the command line.

       -d basename
	      Basename of .pbm file for saving decompressed planes.

       -r basename
	      Basename of .jbg file for saving raw planes

       -i     Suppress display of image records.

       -o     Print file offsets.

       -D level
	      Set Debug level [0].

EXAMPLES

       Decode an OAKT file created by foo2oak.

       $ oakdecode < testpage.oak
       0d(80) 1 OTHER
       0c(64) Wed Nov 05 16:30:50 2003        a07d3   100005  32001e
       0a(80) testpage.pdf
       14(16) (no args)
       28(16) Source=Tray1
       29(80) PaperType=0 UNK8=2,0,0,0, blanks(63)
       2a(32) Copies=1        UNK=0
       2b(32) papercode=25    xwid=4648       ywid=9000       UNK=0
       33(64)
	       u0      u1      w       h       resx    resy    nBits
	       x0      x0      2128    4300    600     600     x1
       15(16) (no args)
	    bih0     w	   h	l0     bih5   dlen  plen unk yOff P subP
       3c(64) 00010000 2176  256  256 58030020  1050  1056 000   64 3 0
		       DL = 0, D = 0, P = 1, - = 0, XY = 2176 x 256
		       L0 = 256, MX = 32, MY = 0
		       Order   = 3  ILEAVE SMID
		       Options = 88  LRLTWO TPDON TPBON
		       1 stripes, 0 layers, 1 planes
       3c(64) 00010000 2176  256  256 58030020  3668  3680 000  320 3 0
       3c(64) 00010000 2176  256  256 58030020  1463  1472 000  640 3 0
       3c(64) 00010000 2176  256  256 58030020  1975  1984 000  896 3 0
       3c(64) 00010000 2176  224  224 58030020  2744  2752 000 1152 3 0
       3c(64) 00010000 2176  256  256 58030020   988	992 000 1440 3 0
       3c(64) 00010000 2176  256  256 58030020  2892  2896 000 1696 3 0
       3c(64) 00010000 2176  256  256 58030020  3634  3648 000 1952 3 0
       3c(64) 00010000 2176  256  256 58030020  3236  3248 000 2208 3 0
       3c(64) 00010000 2176  256  256 58030020  2279  2288 000 2464 3 0
       3c(64) 00010000 2176  256  256 58030020  3746  3760 000 2720 3 0
       3c(64) 00010000 2176  200  200 58030020  2404  2416 000 2976 3 0
       3c(64) 00010000 2176  256  256 58030020  3114  3120 000 3240 3 0
       3c(64) 00010000 2176   96   96 58030020  1142  1152 000 3496 3 0
       3c(64) 00010000 2176  256  256 58030020  2094  2112 000 3752 3 0
       3c(64) 00010000 2176  256  256 58030020  1319  1328 000 4008 3 0
       3c(64) 00010000 2176   36   36 58030020   208	224 000 4264 3 0
       17(16) (no args)
       18(16) UNK=0
       0b(16) (no args)

FILES

       /usr/bin/oakdecode

SEE ALSO

       foo2oak-wrapper(1), foo2oak(1), jbg2pbm(1)

AUTHOR

       Rick Richardson <rick.richardson@comcast.net>
       http://foo2oak.rkkda.com/
All times are GMT -4. The time now is 11:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy