Conditional File Splitting in Unix


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Conditional File Splitting in Unix
# 1  
Old 04-26-2012
Conditional File Splitting in Unix

Hello Guys,

I am new to Unix and i got one requirement where I need to split my file into maximum 5 files on below conditions, if they are splitted into less than 5 then also we are fine.

Columns D value should appear in a single file only and shouldn't continue in other files.


Original file is ordered by Coulmn D (so values will appears as A,B,D,C so on)

Source data

Code:
A|B|C|D|E….
1123|1|20120426|A|A1…
1123|1|20120426|A|A2…
1123|1|20120426|B|B1…
1123|1|20120426|B|B2…
1123|1|20120426|B|B3…
1123|1|20120426|B|B4…
1123|1|20120426|B|B4…
1123|1|20120426|B|B5…
1123|1|20120426|B|B6…
1123|1|20120426|B|B7…
1123|1|20120426|B|B8…
1123|1|20120426|B|B9…
1123|1|20120426|B|B10…
1123|1|20120426|C|C1…
1123|1|20120426|C|C1…
1123|1|20120426|C|C3…
1123|1|20120426|C|C4…
1123|1|20120426|C|C5…
1123|1|20120426|D|D1…
1123|1|20120426|D|D2…

desired output

file No 1

Code:
1123|1|20120426|A|A1…
1123|1|20120426|A|A2…
1123|1|20120426|B|B1…
1123|1|20120426|B|B2…
1123|1|20120426|B|B3…
1123|1|20120426|B|B4…
1123|1|20120426|B|B4…
1123|1|20120426|B|B5…
1123|1|20120426|B|B6…
1123|1|20120426|B|B7…
1123|1|20120426|B|B8…
1123|1|20120426|B|B9…
1123|1|20120426|B|B10…


file No 2

Code:
1123|1|20120426|C|C1…
1123|1|20120426|C|C2…
1123|1|20120426|C|C3…
1123|1|20120426|C|C4…
1123|1|20120426|C|C5…
1123|1|20120426|D|D1…
1123|1|20120426|D|D2…


Last edited by Scrutinizer; 04-26-2012 at 05:03 PM.. Reason: code tags
# 2  
Old 04-26-2012
one way with awk:
Code:
awk -F '|'  '{print $0 > $4}'  inputfilename

You will end up with files named A, B, C ....
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX - 2 tab delimited files, conditional column extraction

Please know that I am very new to unix and trying to learn 'on the job'. I'm only manipulating large tab-delimited files (millions of rows), but I'm stuck and don't know how to proceed with the following. Hoping for some friendly advice :) I have 2 tab-delimited files - with differing column &... (10 Replies)
Discussion started by: GTed
10 Replies

2. Shell Programming and Scripting

Conditional splitting in more file

Dear All, I was wondering in how split one file in multiple file in a conditional way. Briefly, I have a file like this: >Id1 textA >Id2 textBand my outputs file (2 in this case) shoul be: Id1 >Id1 textAId2 >Id2 textBhope you may help me. Best G (5 Replies)
Discussion started by: giuliangiuseppe
5 Replies

3. Shell Programming and Scripting

Help me pls : splitting single file in unix into different files based on data

I have a file in unix with sample data as follows : -------------------------------------------------------------- -------------------------------------------------------------- {30001002|XXparameter|Layout|$ I want this file to be splitted into different files and corresponding to the sample... (54 Replies)
Discussion started by: Ravindra Swan
54 Replies

4. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

5. Shell Programming and Scripting

File splitting and grouping using unix script

Hello All, I have a small problem with file group/splitting and I am trying to get the best way to perform this in unix. I am trying with awk but need some suggestion what would be the best and fastest way to-do it. Here is the problem. I have a fixed length file with filled with product... (4 Replies)
Discussion started by: nandhan11891
4 Replies

6. Shell Programming and Scripting

Conditional Splitting.

hi, I have file with some data delimited by #. For e.g. : RHMS0001#1#ABCD RHMS0002#1#ABCD RHMS0003#1#ABCD RHMS0004#1#ABCD RHMS0005#1#ABCD RHMS0006#1#ABCD RHMS0007#1#ABCD RHMS0001#2#ABCD RHMS0002#2#ABCD RHMS0001#3#ABCD RHMS0004#3#ABCD RHMS0006#3#ABCD (7 Replies)
Discussion started by: pparthiv
7 Replies

7. Shell Programming and Scripting

unix script for conditional execution

Below is my shell script. I am trying to execute two different BTEQ scripts depending on the day of the week. So on a saturday I will execute a certain BTEQ script and on other weekdays I will run the other script. #!/bin/ksh dt=`date +"%a"` if then bteq > final_output <<- EOF .run... (3 Replies)
Discussion started by: Mihirjani
3 Replies

8. Programming

Conditional Compilation based on Environmental Variable in Unix

I want to be able to access an environment variable to control how a program is compiled. So: export MY_VERSN=9 Then ideally, within my C++ code, I would have #if MY_VERSN = 9 iret = FRED9() #else iret = FRED() #endif The way I thought I could do it is that in the script that... (2 Replies)
Discussion started by: BrighterLater
2 Replies

9. Shell Programming and Scripting

splitting a pipe delimited file in unix

Could one of you shad some light on this: I need to split the file by determining the record count and than splitting it up into 4 files. Please note, this is not a fixed record length but rather a "|" delimited file. I am not sure as how to handle reminder/offset for the 4th file. For... (4 Replies)
Discussion started by: ddedic
4 Replies

10. UNIX for Dummies Questions & Answers

splitting a string in unix

i need to split a line using a delimiter, and store it into a array :( (2 Replies)
Discussion started by: lmadhuri
2 Replies
Login or Register to Ask a Question