How to split this txt file into small files? Post: 302840157

Sponsored Content

Top Forums Shell Programming and Scripting How to split this txt file into small files? Post 302840157 by Don Cragun on Sunday 4th of August 2013 04:39:32 PM

08-04-2013

Registered User

The following seems to do what you want:

Code:

awk '
FNR == 1 { next } # Skip header.
last != $1","$2","$3","$4 { # Switch to new file.
        if(fn != "") close(fn) # close previous output file.
        fn = sprintf("s%03d_run%d_condition%d_ACC%d", $1, $2, $3, $4) # Set new output file name.
        last = $1","$2","$3","$4 # Save key for new file.
}
{       print > fn } # Add line to split file.
' a.txt

If you want to run this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

Note that this script is based on a few assumptions:

The split files should not contain a header line; only data lines.
The input lines for a every combination of subject, run, condtion, and ACC are all adjacent in your input file.
If there are any split output files from a previous run, you want to replace those files rather than append the new data to them.

If any of these assumptions are incorrect, this script will need to be modified.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

small script - get sql output & write into txt

Hi, how can I write a small script to run the following statement and output the result into check_result.txt select /*+RULE*/ tapname from typetbl where tapname like 'AA%' and rejectcode=9; Normally, I just type sql and get into SQL>

2. Shell Programming and Scripting

Split a file into 16 small files

Hi I want to split a file that has 'n' number of records into 16 small files. Can some one suggest me how to do this using Unix script? Thanks rrkk

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file?

4. UNIX for Dummies Questions & Answers

How to split a huge file into small pieces (per 2000 columns)?

Dear all, I have a big file:2879(rows)x400,170 (columns) like below. I 'd like to split the file into small pieces:2879(rows)x2000(columns) per file (the last small piece will be 2879x170. So far, I only know how to create one samll piece at one time. But actually I need to repeat this work...

5. Shell Programming and Scripting

KSH script for split a txt file

I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem I have a text file and it has data as follows 1X.....................1234567890123456789T1234598765XT1 (header) 1Z01............(sub HEADER) P100001............ Q1...........

6. Solaris

How to split 10GB file into small Sizes

Hi Team I have one 10 Gb log file I want to split it into say 10 of 1-1Gb file pls share ur experiences how to do this? Thanks in advance,

7. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused:

8. UNIX for Dummies Questions & Answers

Split Every Line In Txt Into Separate Txt File, Named Same As The Line

Hi All Is there a way to export every line into new txt file where by the title of each txt output are same as the line ? I have this txt files containing names: Kandra Vanhooser Rhona Menefee Reynaldo Hutt Houston Rafferty Charmaine Lord Albertine Poucher Juana Maes Mitch Lobel...

9. UNIX for Beginners Questions & Answers

Split a txt file on the basis of line number

I have to split a file containing 100 lines to 5 files say from lines ,1-20 ,21-30 ,31-40 ,51-60 ,61-100 Here is i can do it for 2 file but how to handle it for more than 2 files awk 'NR < 21{ print >> "a"; next } {print >> "b" }' $input_file Please advidse. Thanks

10. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ...

LEARN ABOUT DEBIAN

plan9-split

SPLIT(1)						      General Commands Manual							  SPLIT(1)

NAME

       split - split a file into pieces

SYNOPSIS

       split [ option ...  ] [ file ]

DESCRIPTION

       Split reads file (standard input by default) and writes it in pieces of 1000 lines per output file.  The names of the output files are xaa,
       xab, and so on to xzz.  The options are

       -n n   Split into n-line pieces.

       -l n   Synonym for -n n, a nod to Unix's syntax.

       -e expression
	      File divisions occur at each line that matches a regular expression; see regexp(7).  Multiple -e options may appear.   If  a  subex-
	      pression	of  expression is contained in parentheses (...), the output file name is the portion of the line which matches the subex-
	      pression.

       -f stem
	      Use stem instead of x in output file names.

       -s suffix
	      Append suffix to names identified under -e.

       -x     Exclude the matched input line from the output file.

       -i     Ignore case in option -e; force output file names (excluding the suffix) to lower case.

SOURCE

       /src/cmd/split.c

SEE ALSO

       sed(1), awk(1), grep(1), regexp(7)

																	  SPLIT(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

small script - get sql output & write into txt

Discussion started by: happyv

2. Shell Programming and Scripting

Split a file into 16 small files

Discussion started by: rrkks

3. Shell Programming and Scripting

Split large file and add header and footer to each small files

Discussion started by: ashish4422

4. UNIX for Dummies Questions & Answers

How to split a huge file into small pieces (per 2000 columns)?

Discussion started by: forevertl