How to split this txt file into small files? Post: 302840157

Sponsored Content

Top Forums Shell Programming and Scripting How to split this txt file into small files? Post 302840157 by Don Cragun on Sunday 4th of August 2013 04:39:32 PM

08-04-2013

Registered User

The following seems to do what you want:

Code:

awk '
FNR == 1 { next } # Skip header.
last != $1","$2","$3","$4 { # Switch to new file.
        if(fn != "") close(fn) # close previous output file.
        fn = sprintf("s%03d_run%d_condition%d_ACC%d", $1, $2, $3, $4) # Set new output file name.
        last = $1","$2","$3","$4 # Save key for new file.
}
{       print > fn } # Add line to split file.
' a.txt

If you want to run this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

Note that this script is based on a few assumptions:

The split files should not contain a header line; only data lines.
The input lines for a every combination of subject, run, condtion, and ACC are all adjacent in your input file.
If there are any split output files from a previous run, you want to replace those files rather than append the new data to them.

If any of these assumptions are incorrect, this script will need to be modified.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

small script - get sql output & write into txt

Hi, how can I write a small script to run the following statement and output the result into check_result.txt select /*+RULE*/ tapname from typetbl where tapname like 'AA%' and rejectcode=9; Normally, I just type sql and get into SQL>

2. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file?

4. UNIX for Dummies Questions & Answers

How to split a huge file into small pieces (per 2000 columns)?

Dear all, I have a big file:2879(rows)x400,170 (columns) like below. I 'd like to split the file into small pieces:2879(rows)x2000(columns) per file (the last small piece will be 2879x170. So far, I only know how to create one samll piece at one time. But actually I need to repeat this work...

5. Shell Programming and Scripting

KSH script for split a txt file

I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem I have a text file and it has data as follows 1X.....................1234567890123456789T1234598765XT1 (header) 1Z01............(sub HEADER) P100001............ Q1...........

6. Solaris

How to split 10GB file into small Sizes

Hi Team I have one 10 Gb log file I want to split it into say 10 of 1-1Gb file pls share ur experiences how to do this? Thanks in advance,

7. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused:

8. UNIX for Dummies Questions & Answers

Split Every Line In Txt Into Separate Txt File, Named Same As The Line

Hi All Is there a way to export every line into new txt file where by the title of each txt output are same as the line ? I have this txt files containing names: Kandra Vanhooser Rhona Menefee Reynaldo Hutt Houston Rafferty Charmaine Lord Albertine Poucher Juana Maes Mitch Lobel...

9. UNIX for Beginners Questions & Answers

Split a txt file on the basis of line number

I have to split a file containing 100 lines to 5 files say from lines ,1-20 ,21-30 ,31-40 ,51-60 ,61-100 Here is i can do it for 2 file but how to handle it for more than 2 files awk 'NR < 21{ print >> "a"; next } {print >> "b" }' $input_file Please advidse. Thanks

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ...

LEARN ABOUT DEBIAN

x2sys_merge

X2SYS_MERGE(1gmt)					       Generic Mapping Tools						 X2SYS_MERGE(1gmt)

NAME

       x2sys_merge - Merge an updated COEs tables

SYNOPSIS

       x2sys_merge -Amain_COElist.d -Mnew_COElist.d

DESCRIPTION

       x2sys_merge  will read two crossovers data base and output the contents of the main one updated with the COEs in the second one. The second
       file should only contain updated COEs relatively to the first one.  That is, it MUST NOT contain any new  two  tracks  intersections  (This
       point  is  NOT checked in the code). This program is useful when, for any good reason like file editing NAV correction or whatever, one had
       to recompute only the COEs between the edited files and the rest of the database.

       -A     Specify the file main_COElist.d with the main crossover error data base.

       -M     Specify the file new_COElist.d with the newly computed crossover error data base.

OPTIONS

       No space between the option flag and the associated arguments.

       EXAMPLES
	      To update the main COE_data.txt with the new COEs estimations saved in the smaller COE_fresh.txt, try

	      x2sys_merge -ACOE_data.txt -MCOE_fresh.txt > COE_updated.txt

SEE ALSO

       x2sys_binlist(1), x2sys_cross(1), x2sys_datalist(1), x2sys_get(1), x2sys_init(1), x2sys_list(1), x2sys_put(1), x2sys_report(1)

GMT 4.5.7							    15 Jul 2011 						 X2SYS_MERGE(1gmt)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

small script - get sql output & write into txt

Discussion started by: happyv

2. Shell Programming and Scripting

Split a file into 16 small files

Discussion started by: rrkks

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

Discussion started by: ashish4422

4. UNIX for Dummies Questions & Answers

How to split a huge file into small pieces (per 2000 columns)?

Discussion started by: forevertl