How to split this txt file into small files?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to split this txt file into small files?
# 1  
Old 08-03-2013
How to split this txt file into small files?

Dear shell experts,

I would like to spilt a txt file into small ones. However, I did not know how to program use shell. If someone could help, it is greatly appreciated!

Specifically, I supposed there is file named A.txt. The content of the file likes this:

Code:
Subject	run	condtion	ACC	time 	duration	weight
2	1	1	0	122.6 	2	1
2	1	1	0	144.8 	2	1
2	1	1	1	132.6 	2	1
2	1	1	1	182.6 	2	1
2	1	1	1	198.6 	2	1
2	1	1	1	230.6 	2	1
2	1	1	1	308.6 	2	1
2	1	1	1	368.6 	2	1
2	1	1	1	382.6 	2	1
2	1	1	1	410.6 	2	1
2	1	2	0	294.8 	2	1
2	1	2	0	394.8 	2	1
2	1	2	1	66.6 	        2	1
2	1	2	1	78.6 	        2	1
2	1	2	1	158.6 	2	1
2	1	2	1	207.1 	2	1

..........................
I want to split the last three columns into many small ones. if the fourth column, namely "ACC' is zero, I want to put them into a txt file, named "s00x_run1_condition1_ACC0". If the fourth column, namely "ACC' is one, I want to put them into a txt file, named "s00x_run1_condition1_ACC1"

For example, in the first two rows, the ACC column is zero, so I put the first and the second column into one txt file, and named "s00x_run1_condition1_ACC1", there are 50 subjects and every subject have 6 runs, and every runs have 3 conditions.

Although I can do those manually, it is a time-consuming task. Thus, if anybody could help, it will be greatly appreciated!

Thank you very much in advance!

Last edited by Don Cragun; 08-04-2013 at 02:04 AM.. Reason: Add CODE tags.
# 2  
Old 08-04-2013
Your description is very vague.

Please add a few lines to your sample input for a subject with a 2-digit subject number, and then show us the exact names of the output files you want to have produced from that input and the contents that should be saved in each of those files.

You show the name of the output files as s00x_run1_condition1_ACC1. Does that mean that subject 25 should have filename s00x_run1_condition1_ACC1, s0025_run1_condition1_ACC1, or s025_run1_condition1_ACC1?

What field delimiter do you want between output fields? Your input has tabs as separators for most fields; but some use a space and a tab; and some use a space, a tab, and several more spaces?

Do you want to throw away the data in the 1st four fields in the new, split files; or do you want to preserve the current lines but just split them into new files?
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 08-04-2013
Hi

Thank you for your generous help! I am sorry that I did not make it clear. The original post may difficult to understand. So, I made some changes. I make the file into four columns like this:

Code:
name	Onsetime	duration	weight
s002_run1_fng	122.6 	2	1
s002_run1_fng	144.8 	2	1
s002_run1_fyg	132.6 	2	1
s002_run1_fyg	182.6 	2	1
s002_run1_fyg	198.6 	2	1
s002_run1_fyg	230.6 	2	1
s002_run1_fyg	308.6 	2	1
s002_run1_fyg	368.6 	2	1
s002_run1_fyg	382.6 	2	1
s002_run1_fyg	410.6 	2	1
s002_run1_fnl	294.8 	2	1
s002_run1_fnl	394.8 	2	1
s002_run1_fyl	66.6 	2	1
s002_run1_fyl	78.6 	2	1
s002_run1_fyl	158.6 	2	1
s002_run1_fyl	207.1 	2	1
s002_run1_fyl	257.4 	2	1
s002_run1_fyl	269.2 	2	1
s002_run1_fyl	319.2 	2	1
s002_run1_fyl	327.3 	2	1
s002_run1_fnn	52.8 	2	1
s002_run1_fnn	280.7 	2	1
s002_run1_fnn	350.8 	2	1
s002_run1_fyn	96.6 	2	1
s002_run1_fyn	110.6 	2	1
s002_run1_fyn	169.3 	2	1

I also attached the file. I would like to split the file into new txt files, the first column will be the name of the new file, and the second to the fourth column is the content of the new file. If the first column is the same, the second to the fourth column will be composed a new file. For example, the first file I want to is
Code:
122.6 	2	1
144.8 	2	1

and the new file name would be "s002_run1_fng"

How to code using shell ?

Thank you! Your help is greatly appreciated!


feedback.txt

Last edited by psychmyluo; 08-04-2013 at 05:39 PM.. Reason: Add CODE tags
# 4  
Old 08-04-2013
The following seems to do what you want:
Code:
awk '
FNR == 1 { next } # Skip header.
last != $1","$2","$3","$4 { # Switch to new file.
        if(fn != "") close(fn) # close previous output file.
        fn = sprintf("s%03d_run%d_condition%d_ACC%d", $1, $2, $3, $4) # Set new output file name.
        last = $1","$2","$3","$4 # Save key for new file.
}
{       print > fn } # Add line to split file.
' a.txt

If you want to run this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

Note that this script is based on a few assumptions:
  1. The split files should not contain a header line; only data lines.
  2. The input lines for a every combination of subject, run, condtion, and ACC are all adjacent in your input file.
  3. If there are any split output files from a previous run, you want to replace those files rather than append the new data to them.
If any of these assumptions are incorrect, this script will need to be modified.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies

2. UNIX for Beginners Questions & Answers

Split a txt file on the basis of line number

I have to split a file containing 100 lines to 5 files say from lines ,1-20 ,21-30 ,31-40 ,51-60 ,61-100 Here is i can do it for 2 file but how to handle it for more than 2 files awk 'NR < 21{ print >> "a"; next } {print >> "b" }' $input_file Please advidse. Thanks (4 Replies)
Discussion started by: abhaydas
4 Replies

3. UNIX for Dummies Questions & Answers

Split Every Line In Txt Into Separate Txt File, Named Same As The Line

Hi All Is there a way to export every line into new txt file where by the title of each txt output are same as the line ? I have this txt files containing names: Kandra Vanhooser Rhona Menefee Reynaldo Hutt Houston Rafferty Charmaine Lord Albertine Poucher Juana Maes Mitch Lobel... (2 Replies)
Discussion started by: Nexeu
2 Replies

4. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

5. Solaris

How to split 10GB file into small Sizes

Hi Team I have one 10 Gb log file I want to split it into say 10 of 1-1Gb file pls share ur experiences how to do this? Thanks in advance, (3 Replies)
Discussion started by: zimmyyash
3 Replies

6. Shell Programming and Scripting

KSH script for split a txt file

I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem I have a text file and it has data as follows 1X.....................1234567890123456789T1234598765XT1 (header) 1Z01............(sub HEADER) P100001............ Q1........... (4 Replies)
Discussion started by: ask.chowhan
4 Replies

7. UNIX for Dummies Questions & Answers

How to split a huge file into small pieces (per 2000 columns)?

Dear all, I have a big file:2879(rows)x400,170 (columns) like below. I 'd like to split the file into small pieces:2879(rows)x2000(columns) per file (the last small piece will be 2879x170. So far, I only know how to create one samll piece at one time. But actually I need to repeat this work... (6 Replies)
Discussion started by: forevertl
6 Replies

8. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

9. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk (10 Replies)
Discussion started by: rrkks
10 Replies

10. Shell Programming and Scripting

small script - get sql output & write into txt

Hi, how can I write a small script to run the following statement and output the result into check_result.txt select /*+RULE*/ tapname from typetbl where tapname like 'AA%' and rejectcode=9; Normally, I just type sql and get into SQL> (2 Replies)
Discussion started by: happyv
2 Replies
Login or Register to Ask a Question