Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Splitting a file based on record sin another file Post 302194492 by era on Tuesday 13th of May 2008 03:26:13 AM
Old 05-13-2008
If your grep can read patterns from a file (like GNU grep), try something like this.

Code:
cut -c1-5 file2 >patterns
grep -f patterns file1 >matches
grep -f patterns -v file1 >nonmatches

There is some room for improvement, relating to making the patterns always match at beginning of line, etc. You could replace the first line with something like

Code:
sed 's/^\(.....\).*/^"\1",/' file2 >patterns

This adds the double quotes around the search string and adds a comma behind it and the special character "^" before it, which means match only at beginning of line.

Last edited by era; 05-13-2008 at 04:28 AM.. Reason: Explain revised patterns
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

splitting a record and adding a record to a file

Hi, I am new to UNIX scripting and woiuld appreicate your help... Input file contains only one (but long) record: aaaaabbbbbcccccddddd..... Desired file: NEW RECORD #new record (hardcoded) added as first record - its length is irrelevant# aaaaa bbbbb ccccc ddddd ... ... ... (1 Reply)
Discussion started by: rsolap
1 Replies

2. Shell Programming and Scripting

Splitting a file based on the records in another file

All, We receive a file with a large no of records (records can vary) and we have to split it into two files based on another file. e.g. File1: UHDR 2008112 "25187","00000022","00",21-APR-1991,"" ,"D",-000000519,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ... (2 Replies)
Discussion started by: er_ashu
2 Replies

3. Shell Programming and Scripting

Splitting the file based on logic

Hello I have a requirement where i need to split the Input fixed width file which contains multiple invoices into multiple files with 2 invoices per file. Each invoice can be identified by its first line's second character which is "H" and sixth character is " " space and the invoice would... (10 Replies)
Discussion started by: dsdev_123
10 Replies

4. Shell Programming and Scripting

Splitting a file based on two patterns

Hi there, I've an input file as follows: *START 1001 a1 1002 a2 1003 a3 1004 a4 *END *START 1001 b1 1002 b2 1004 b4 *END *START 1001 c1 1004 c4 *END (6 Replies)
Discussion started by: kbirde
6 Replies

5. Shell Programming and Scripting

Splitting a file based on context.

I have file as shown below. Would like to split the file based on the context of data. Like, split the content between "---- XXX Info ----" and " ---- YYY Info ----" to a file. When I try using below command, 2nd file contains all the info starting after first "---- YYYY Info ----" instance.... (8 Replies)
Discussion started by: webkid
8 Replies

6. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

7. UNIX for Dummies Questions & Answers

Extracting data from one file, based on another file (splitting)

Dear All, I have two files but want to extract data from one based on another... can you please help me file 1 David Tom Ellen and file 2 David|0010|testnamez|resultsz David|0004|testnamex|resultsx Tom|0010|testnamez|resultsz Tom|0004|testnamex|resultsx Ellen|0010|testnamez|resultsz... (12 Replies)
Discussion started by: A-V
12 Replies

8. Shell Programming and Scripting

Keeping record of file 2 based on a reference file 1 in awk

I have 2 input files (tab separated): file1: make_A 1990 foo bar make_B 2010 this that make_C 2004 these those file2: make_X 1970 1995 ref_1:43 ref_2:65 make_A 1970 1995 ref_1:4 ref_2:21 ref_3:18 make_A 1980 2002 ref_1:7 ref_2:7 ref_3:0 ... (2 Replies)
Discussion started by: beca123456
2 Replies

9. UNIX for Beginners Questions & Answers

Splitting a file based on a pattern

Hi All, I am having a problem. I tried to extract the chunk of data and tried to fix I am not able to. Any help please Basically I need to remove the for , values after K, this is how it is now A,, B, C,C, D,D, 12/04/10,12/04/10, K,1,1,1,1,0,3.0, K,1,1,1,2,0,4.0,... (2 Replies)
Discussion started by: arunkumar_mca
2 Replies

10. UNIX for Beginners Questions & Answers

Splitting the file based on two fields - Fixed length file

Hi , I am having a scenario where I need to split the file based on two field values. The file is a fixed length file. ex: AA0998703000000000000190510095350019500010005101980301 K 0998703000000000000190510095351019500020005101480 ... (4 Replies)
Discussion started by: saj
4 Replies
grep(1) 						      General Commands Manual							   grep(1)

Name
       grep, egrep, fgrep - search file for regular expression

Syntax
       grep [option...] expression [file...]

       egrep [option...] [expression] [file...]

       fgrep [option...] [strings] [file]

Description
       Commands  of  the family search the input files (standard input default) for lines matching a pattern.  Normally, each line found is copied
       to the standard output.

       The command patterns are limited regular expressions in the style of which uses a compact nondeterministic algorithm.  The command patterns
       are  full  regular  expressions.  The command uses a fast deterministic algorithm that sometimes needs exponential space.  The command pat-
       terns are fixed strings.  The command is fast and compact.

       In all cases the file name is shown if there is more than one input file.  Take care when using the characters $ * [ ^ | ( ) and   in  the
       expression because they are also meaningful to the Shell.  It is safest to enclose the entire expression argument in single quotes ' '.

       The command searches for lines that contain one of the (new line-separated) strings.

       The command accepts extended regular expressions.  In the following description `character' excludes new line:

	      A  followed by a single character other than new line matches that character.

	      The character ^ matches the beginning of a line.

	      The character $ matches the end of a line.

	      A .  (dot) matches any character.

	      A single character not otherwise endowed with special meaning matches that character.

	      A  string  enclosed in brackets [] matches any single character from the string.	Ranges of ASCII character codes may be abbreviated
	      as in `a-z0-9'.  A ] may occur only as the first character of the string.  A literal - must be placed where it can't be mistaken	as
	      a range indicator.

	      A  regular  expression  followed	by  an	* (asterisk) matches a sequence of 0 or more matches of the regular expression.  A regular
	      expression followed by a + (plus) matches a sequence of 1 or more matches of the regular expression.  A regular expression  followed
	      by a ? (question mark) matches a sequence of 0 or 1 matches of the regular expression.

	      Two regular expressions concatenated match a match of the first followed by a match of the second.

	      Two regular expressions separated by | or new line match either a match for the first or a match for the second.

	      A regular expression enclosed in parentheses matches a match for the regular expression.

       The  order  of  precedence  of  operators at the same parenthesis level is the following:  [], then *+?, then concatenation, then | and new
       line.

Options
       -b	   Precedes each output line with its block number.  This is sometimes useful in locating disk block numbers by context.

       -c	   Produces count of matching lines only.

       -e expression
		   Uses next argument as expression that begins with a minus (-).

       -f file	   Takes regular expression (egrep) or string list (fgrep) from file.

       -i	   Considers upper and lowercase letter identical in making comparisons and only).

       -l	   Lists files with matching lines only once, separated by a new line.

       -n	   Precedes each matching line with its line number.

       -s	   Silent mode and nothing is printed (except error messages).	This is useful for checking the error status (see DIAGNOSTICS).

       -v	   Displays all lines that do not match specified expression.

       -w	   Searches for an expression as for a word (as if surrounded by `<' and `>').  For further information, see only.

       -x	   Prints exact lines matched in their entirety only).

Restrictions
       Lines are limited to 256 characters; longer lines are truncated.

Diagnostics
       Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files.

See Also
       ex(1), sed(1), sh(1)

																	   grep(1)
All times are GMT -4. The time now is 08:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy