Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Please Help:Need to Split the file into mutliple files depends on the KEY field value
Top Forums UNIX for Dummies Questions & Answers Please Help:Need to Split the file into mutliple files depends on the KEY field value Post 302186350 by era on Thursday 17th of April 2008 02:55:48 AM
Old 04-17-2008
Different values in different files, or multiple values in multiple files, or different sets of values in different sets of files?

Code:
egrep 'first|second|third' file1 file2 file3 >have
egrep -v 'first|second|third' file1 file2 file3 >have.not

If the value is different for each file, you need a loop.

Code:
>have; >have.not  # make sure they're empty before we start
while read file value; do
  grep    "$value" "$file" >>have
  grep -v "$value" "$file" >>have.not
done <<HERE
file1  first
file2  second.*rege[xz]magic$
file3  how sweet I roam from field to field
HERE

This is to illustrate that you can have fairly complex "value" fields in the here document. You probably don't need that flexibility for this particular problem, but if you do, it's there. (And if you need to search for stuff which has special meaning in regular expressions, such as asterisks or dollar signs, you need to escape them, or something.)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split file based on field

Hi I have a large file 2.6 million records and I am trying to split the file based on last column. I am doing awk -F"|" '{ print > $NF }' filename1 After around 1000 splits it gives me a error awk: can't open file 3332332423 input record number 1068, file filename1 source... (6 Replies)
Discussion started by: s_adu
6 Replies

2. Shell Programming and Scripting

split large file based on field criteria

I have a file containing date/time sorted data of the form ... 2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1 2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1 2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0 2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies

3. Shell Programming and Scripting

Split file when the key field change !

Hello, I have the following example data file: Rv.Global_Sk,1077.160523,D,16/09/2011 Rv.Global_Sk,1077.08098,D,17/09/2011 Rv.Global_Sk,1077.001445,D,18/09/2011 Rv.Global_Sk,1072.660733,D,19/09/2011 Rv.Global_Sk,1070.381557,D,20/09/2011 Rv.Global_Sk,1071.971747,D,21/09/2011... (4 Replies)
Discussion started by: csierra
4 Replies

4. Shell Programming and Scripting

awk to split one field and print the last two fields within the split part.

Hello; I have a file consists of 4 columns separated by tab. The problem is the third fields. Some of the them are very long but can be split by the vertical bar "|". Also some of them do not contain the string "UniProt", but I could ignore it at this moment, and sort the file afterwards. Here is... (5 Replies)
Discussion started by: yifangt
5 Replies

5. Shell Programming and Scripting

Split a file into multiple files based on field value

Hi, I've one requirement. I have to split one comma delimited file into multiple files based on one of the column values. How can I achieve this Unix Here is the sample data. In this case I have split the files based on date column(c4) Input file c1,c2,c3,c4,c5... (1 Reply)
Discussion started by: manasvi24
1 Replies

6. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

7. UNIX for Dummies Questions & Answers

Split a file into parts only if the first field is different

Hi, I have a file like this: aaa 123 aaa 223 aaa 225 bbb 332 bbb 423 bbb 6755 bbb 324 ccc 112 ccc 234 ccc 897 Which I need to split into several files, something like split -l 3 but the way that the lines with the same names would only go into one file: (7 Replies)
Discussion started by: coppuca
7 Replies

8. Shell Programming and Scripting

Split file based on a column/field value

Hi All, I have a requirement to split file into 2 sets of file. Below is a sample data of the file AU;PTN;24EX;25-AUG-14;AU;123;SE;123;Test NN;;;;ASD; AU;PTN;24EX;25-AUG-14;AU;456;SE;456;Test NN;;;;ASD; AU;PTN;24EX;25-AUG-14;AU;147;SE;147;Test NN;;;;ASD;... (6 Replies)
Discussion started by: galaxy_rocky
6 Replies

9. Shell Programming and Scripting

EBCDIC File Split Based On Record Key

I was wondering if anyone could explain to me how to split a variable length EBCDIC file into seperate files based on the record key. I have the COBOL layout, and so I need to split the file into 13 different EBCDIC files so that I can run each one through a C++ converter I have, and get the... (11 Replies)
Discussion started by: hanshot1stx
11 Replies

10. Shell Programming and Scripting

Update a specific field in file with Variable value based on other Key Word

I have an input file with A=xyz B=pqr I would want the value in Second Field (xyz or pqr) updated with a value present in Shell Variable based on the value passed in the first field. (A or B ) while read line do NEW_VALUE = `some functionality done on $line` If $line=First Field-... (1 Reply)
Discussion started by: infernalhell
1 Replies

LEARN ABOUT PLAN9

split

SPLIT(1)						      General Commands Manual							  SPLIT(1)

NAME

       split - split a file into pieces

SYNOPSIS

       split [ option ...  ] [ file ]

DESCRIPTION

       Split reads file (standard input by default) and writes it in pieces of 1000 lines per output file.  The names of the output files are xaa,
       xab, and so on to xzz.  The options are

       -n     Split into n-line pieces.

       -e expression
	      File divisions occur at each line that matches a regular expression; see regexp(6).  Multiple -e options may appear.   If  a  subex-
	      pression	of  expression is contained in parentheses (...), the output file name is the portion of the line which matches the subex-
	      pression.

       -f stem
	      Use stem instead of x in output file names.

       -s suffix
	      Append suffix to names identified under -e.

       -x     Exclude the matched input line from the output file.

       -i     Ignore case in option -e; force output file names (excluding the suffix) to lower case.

SOURCE

       /sys/src/cmd/split.c

SEE ALSO

       sed(1), awk(1) grep(1), regexp(6)

																	  SPLIT(1)
All times are GMT -4. The time now is 03:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy