Splitting a file by arbitrary values


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting a file by arbitrary values
# 1  
Old 12-04-2009
Splitting a file by arbitrary values

How do I split a file into unequal line number segments? For example, I have a 100,000 line file that I need to split into 4 or 5 segments and the segments are not uniform, so split will not work in this case. The five segments might be: 27,000 44,512 13,542 2,344 & 12,602.

I can accomplish this by copying sections of files doing:
Code:
sed -n '1,27000p' filename
sed -n '27001,71513p' filename
sed -n '71514,85055p' filename
etc

I am wondering what the more elegant method(s) would be

Thanks

Last edited by zaxxon; 12-04-2009 at 08:43 AM.. Reason: code tags
# 2  
Old 12-04-2009
You can do something like this, the created files are file1, file2 etc.

Code:
awk '
NR==1{f="file1"}
NR==27001{f="file2"}
NR==71514{f="file3"}
{print > f}
' file

# 3  
Old 12-04-2009
Yes.

Just to add that there is also csplit. While almost completely incomprehensible :-), it is more flexible than split.
# 4  
Old 12-04-2009
Scottn - can you demonstrate how csplit would work for the example given?

Franklin52 - That is a valid awk alternative, but it is not much more elegant than the sed method.

Thanks for the ideas though.
# 5  
Old 12-04-2009
I suppose the simplest form would be
Code:
csplit -f filename. filename 27001 71514 85056

If 85056 goes past the end of your file, leave it off.

Check the man page.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Taking key values from one file and extracting values from another file

Hi, I have two files with values in both. File1: cat 2 3 dog 4 5 elephant 6 7 camel 2 3 File2: ----+--gkf;ajf= ---+---- +----- cat -------=----+ 3 | 4 ----- dog ------++-- 5 | 9 ----++-- elephant | 5 | 7 ---++ camel ------ ++++_---- || 8 | 9 I want the final file as: cat 4... (1 Reply)
Discussion started by: npatwardhan
1 Replies

2. Shell Programming and Scripting

Splitting the numeric vs alpha values in a column to distinct columns

How could i take an input file and split the numeric values from the alpha values (123 vs abc) to distinc columns, and if the source is blank to keep it blank (null) in both of the new columns: So if the source file had a column like: Value: |1 | |2.3| | | |No| I would... (7 Replies)
Discussion started by: driftlogic
7 Replies

3. Shell Programming and Scripting

awk file to read values from Db2 table replacing hard coded values

Hi, I want to replace a chain of if-else statement in an old AWK file with values from Db2 table or CSV file. The part of code is below... if (start_new_rec=="true"){ exclude_user="false"; user=toupper($6); match(user, "XXXXX."); if (RSTART ==2 ) { ... (9 Replies)
Discussion started by: asandy1234
9 Replies

4. Shell Programming and Scripting

Read record from the text file contain multiple separated values & assign those values to variables

I have a file containing multiple values, some of them are pipe separated which are to be read as separate values and some of them are single value all are these need to store in variables. I need to read this file which is an input to my script Config.txt file name, first path, second... (7 Replies)
Discussion started by: ketanraut
7 Replies

5. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

6. Shell Programming and Scripting

Splitting record into multiple records by appending values from an input field (AWK)

Hello, For the input file, I am trying to split those records which have multiple values seperated by '|' in the last input field, into multiple records and each record corresponds to the common input fields + one of the value from the last field. I was trying with an example on this forum... (4 Replies)
Discussion started by: imtiaz99
4 Replies

7. Shell Programming and Scripting

Splitting file based on column values

Hi all, I have a file (say file.txt) which contains comma-separated rows. Each row has seven columns. Only column 4 or 5 (not both) can have empty values like "" in each line. Sample lines So, now i want all the rows that have column 4 as "" go in file1.txt and all the rows that have column... (8 Replies)
Discussion started by: jakSun8
8 Replies

8. Shell Programming and Scripting

Splitting a file in to multiple files and passing each individual file to a command

I have an input file with contents like: MainFile.dat: 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 ... (4 Replies)
Discussion started by: rkrish
4 Replies

9. Shell Programming and Scripting

splitting single column values into text and number component

Hey guys, I have a column that consists of string and integer values without a distinctive deliminator, looking like this... 7ASA 14LAL 245FOO 656MOM 87577DAD ... I want to split the column into two columns, one containing the numbers and one containing the text part. edit: numbers... (3 Replies)
Discussion started by: origamisven
3 Replies

10. Shell Programming and Scripting

Splitting comma separated values into an array

I'm attempting to create a KSH array out of a string like this: ",,,value1,value2,," I have created the array but I only get two elements, one for value1 and one for value2. I have ended up with something like this but I don't like it: set -A JUNK xx=0 for i in $(print ",,,value1,value2,,"... (3 Replies)
Discussion started by: tmarikle
3 Replies
Login or Register to Ask a Question