Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to split a data file into separate files with the file names depending upon a column's value?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to split a data file into separate files with the file names depending upon a column's value?
# 1  
Old 09-25-2011
Question How to split a data file into separate files with the file names depending upon a column's value?
Hi,
I have a data file xyz.dat similar to the one given below,
Code:
2345|98|809||x|969|0
2345|98|809||y|0|537
2345|97|809||x|544|0
2345|97|809||y|0|651
9685|98|809||x|321|0
9685|98|809||y|0|357
9685|98|709||x|687|0
9685|98|709||y|0|234
2315|98|809||x|564|0
2315|98|809||y|0|537
2315|98|809||x|523|0
2315|98|809||y|0|535

I need to split the file into seperate file with the condition that :
1) All the rows which has the same value in column 1 should be in one file.
2) The name of the file should be starting with the column 1's value.

Code:
ie:
2345_xyz.dat should be created and it should contain only the below data:
2345|98|809||y|0|537
2345|97|809||x|544|0
2345|97|809||y|0|651

9685_xyz.dat should be created and it should contain only the below data:
9685|98|809||x|321|0
9685|98|809||y|0|357
9685|98|709||x|687|0
9685|98|709||y|0|234

2315_xyz.dat should be created and it should contain only the below data:
2315|98|809||x|564|0
2315|98|809||y|0|537
2315|98|809||x|523|0
2315|98|809||y|0|535

Also the main data file xyz.dat should not be removed.
# 2  
Old 09-25-2011
Hi, try this:
Code:
awk -F\| '{n=$1"_xyz.dat"}f!=n{close(f);f=n}{print>f}' file

Last edited by Scrutinizer; 09-25-2011 at 08:43 AM.. Reason: Changed it, otherwise open files still would not get closed and there might be too many open files
This User Gave Thanks to Scrutinizer For This Post:
nithins007
# 3  
Old 09-25-2011
Code:
awk -F\| '{print > $1"_xyz.dat"}' inputfile

This User Gave Thanks to itkamaraj For This Post:
nithins007
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to separate data coming in one column of CSV file?

I am running an ISQL command on Sybase DB and getting output of a query in an CSV file. The issue is that all the data comes in to the same column, i want them to be separated in different columns. SQL_COMMAND=command.sql file=file.txt formatFile=formatFile.txt report=report.csv echo... (1 Reply)
Discussion started by: Sharma331
1 Replies

2. UNIX for Dummies Questions & Answers

Split larger files into smaller ones with Column names

Hi, I have one large files of 100000 rows with header column. Eg: Emp Code, Emp Name 101,xxx 102,YYY 103,zzz ... ... I want to split the files into smaller files with only 30000 rows each..File 1,2 and 3 must have 30000 rows and file 4 must contain 10000 rows. But the column... (1 Reply)
Discussion started by: Nivas
1 Replies

3. Shell Programming and Scripting

Split file depending on Column Value

Hi , my file look likes below , cat file.csv 12/09/2014,50,5,0,300 12/09/2014, ,5,0,300 12/09/2014,50,,,300 i need to split file , the first one contains values (2nd column is 50 , 3rd and fourth column is null ) the second file contains all others firstfile ... (2 Replies)
Discussion started by: ubaisalih
2 Replies

4. Shell Programming and Scripting

How to Split File to 2 depending on condition?

Hi , cat myfile.txt ! 3100.2.0.5 ! 3100.2.22.4 ! 3100.2.30.33 ! 3100.2.4.1 ! ! 3100.2.0.5 ! 3100.2.22.4 ! 3100.2.22.11 ! 3100.2.4.1 ! ! 3100.2.0.5 ! 3100.2.2.50 ! 3100.2.22.11 ! 3100.2.4.1 ! ! 3100.2.0.5 ! 3100.2.22.4 ! 3100.2.30.33 ! 3100.2.4.1 ! ! 3100.2.0.5 ! 3100.2.22.4 !... (6 Replies)
Discussion started by: OTNA
6 Replies

5. Shell Programming and Scripting

Divide data with specific column values into separate files

hello! i need a little help from you :) ... i need to split a file into separate files depending on two conditions using scripting. The file has no delimiters. The conditions are col 17 = "P" and col 81 = "*", this will go to one output file; col 17 = "R" and col 81 = " ". Here is an example. ... (3 Replies)
Discussion started by: chanclitas
3 Replies

6. Shell Programming and Scripting

find specific file names and execute a command depending on file's name

Hi, As a newbie, I'm desperate ro make my shell script work. I'd like a script which checks all the files in a directory, check the file name, if the file name ends with "extracted", store it in a variable, if it has a suffix of ".roi" stores in another variable. I'm going to use these two... (3 Replies)
Discussion started by: armando110
3 Replies

7. UNIX for Dummies Questions & Answers

How to separate a single column file into files of the same size (i.e. number of rows)?

I have a text file with 1,000,000 rows (It is a single column text file of numbers). I would like to separate the text file into 100 files of equal size (i.e. number of rows). The first file will contain the first 10,000 rows, the second row will contain the second 10,000 rows (rows 10,001-20,000)... (2 Replies)
Discussion started by: evelibertine
2 Replies

8. Shell Programming and Scripting

Using AWK to separate data from a large XML file into multiple files

I have a 500 MB XML file from a FileMaker database export, it's formatted horribly (no line breaks at all). The node structure is basically <FMPXMLRESULT> <METADATA> <FIELD att="............." id="..."/> </METADATA> <RESULTSET FOUND="1763457"> <ROW att="....." etc="...."> ... (16 Replies)
Discussion started by: JRy
16 Replies

9. Shell Programming and Scripting

Split file into multiple files depending upon first 4 digits

Hi All, I have a file like below: 1016D"ddd","343","1299" 1016D"ddd","3564","1299" 1016D"ddd","3297","1393" 1016D"ddd","32989","1527" 1016D"ddd","346498","1652" 2312D"ddd","3269","1652" 2312D"ddd","328","1652" 2312D"ddd","2224","2100" 3444D"ddd","252","2100" 3444D"ddd","2619","2100"... (4 Replies)
Discussion started by: deepakgang
4 Replies

10. Shell Programming and Scripting

split file depending on content

Hi, I have a file which contains records of data. I need to split the file into multiple files depending upon the value of last field. How do i read the last field of each record in the file??? Regards, Chaitrali (4 Replies)
Discussion started by: Chaitrali
4 Replies
Login or Register to Ask a Question