Problem in split command


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Problem in split command
# 1  
Old 06-03-2005
Problem in split command

I want to split a file containing millions of records.
I am issuing the command
split -l 20000 filename which will split the file in 20K records each.
It works fine except in some files, data after one particular field is lost( the field with space).
Say the record is
C||121.5||N|131.20|13.5|0|0|0|9.2|4.3|||A||12312007|THIS FIELD|0|0|0||T|PM
The split file contains record as
C||121.5||N|131.20|13.5|0|0|0|9.2|4.3|||A||12312007|THIS
Is this due to spaces?
Any solutions?
# 2  
Old 06-03-2005
Lightbulb Not related to question but :-

split -l 20000 file : - will split the input file into number of files each containing 20000 lines and not 20KB files .

Use split -b k 20 file to split the file into multiple 20k files .


Rahul Smilie
# 3  
Old 06-03-2005
wrong syntax:
it is split -b 20k file.tar.whatever :-)
gP
# 4  
Old 06-03-2005
I think your wording got you some interesting answers.

split is not normally affected by spaces, to answer your question. In your use it only pays attention to \n (newline) characters or control characters like vertical tab. It may be a data issue, like embedded control characters.

Try using od on the offending line in the original source file. Look for junk characters.
Code:
head -offendingline# filename | tail -1 | od -x

# 5  
Old 06-06-2005
Well, thanks for the responses
I need to split the file in 20K (20,000) records, not in 20K size
Jim I really dont understand what that od does

Last edited by superprogrammer; 06-06-2005 at 02:45 AM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with Split Command

Hi All, I have a txt file which I would like to partition into 2 separate output files. I would like to partition the odd or even groups of 4 lines from the txt file. So I would like lines 1-4 to go to file1, and lines 5-8 to go to file2, and so on until the whole txt file is divided into two... (1 Reply)
Discussion started by: landrjos
1 Replies

2. UNIX for Beginners Questions & Answers

Urgent..!!Split command

Hi All, I want to split the file after size gets above 100kb. So I am using below command. split -b 100kb File.txt Test But after first file, my record is breaking as in middle of the record, size of file is getting above 100kb. So after splitting half record is in one file and half... (1 Reply)
Discussion started by: Amey Dixit
1 Replies

3. UNIX for Dummies Questions & Answers

Split command

Hi I have a sequence which looks like this # PH01000000 PH01000000G0240 P.he_genemodel_v1.0 CDS 120721 121773 . - . ID=PH01000000G0240.CDS;Parent=PH01000000G0240 PH01000001G0190 P.he_genemodel_v1.0 mRA 136867 137309 . - . ID=PH01000001G0190.mRNA;Parent=PH01000001G0190... (7 Replies)
Discussion started by: siya@
7 Replies

4. UNIX for Dummies Questions & Answers

change split with another command

hi all, i have problem with my script in unix ...i have script with split -d (--numeric-suffixes) in linux its working but in solaris machine the option -d isn't have so how to i can change split -d (this output) will same in output solaris can i change with awk and how do that thx before (2 Replies)
Discussion started by: zvtral
2 Replies

5. Shell Programming and Scripting

problem with awk for file split

Hi all, i have a .ksh script which is, among other stuff, splitting a file and saveing the filenames into variables for further processing: # file split before ftp and put result filenames into variables if ]; then awk '{close(f);f=$1}{sub("^","");print > f".TXT"}' $_ftpfile set B*.TXT... (0 Replies)
Discussion started by: spidermike
0 Replies

6. UNIX for Advanced & Expert Users

split command

./myapp | split -b 10m -d -a 1 - "myappLog" here split command is reading the input from the output of myapp and it will write the text in to file where in each file size is 10MB and it will create upto 10 files. I have observed split is flushing the data for every 4096 bytes. if my... (7 Replies)
Discussion started by: arv600
7 Replies

7. Shell Programming and Scripting

split file problem

Hi All, I have a parent file with 10 million records. I want to split the parent file in to child files. Each child file contains 5000 records. I am using the following command for splitting: split -5000 parentfile.txt childfile.1 It will split the Parent file as... (7 Replies)
Discussion started by: hanu_oracle
7 Replies

8. UNIX for Dummies Questions & Answers

filenames from split command

Is there an option or a way with the split command to rename the partitioned files with a counter. For example, can the files testaa, testab, testac be renamed to test1, test2, test3 from the split command without explicilty renaming files. Thanks, - CB (3 Replies)
Discussion started by: ChicagoBlues
3 Replies

9. UNIX for Advanced & Expert Users

Split Command options

HI! All iam using Split command to split a large .txt file in to smaller files, The syntax iam using split -25000 Product.txt iam getting four output files but not in .txt format but in some other format , when i checked the properties the Type of the output files is Type can any... (7 Replies)
Discussion started by: mohdtausifsh
7 Replies

10. Shell Programming and Scripting

Split command

Can anyone tell me what this command will do? split -b$SPLITSIZE - $file1 < $file2 Will it split file1 or file2? Please explain. Malay (1 Reply)
Discussion started by: malaymaru
1 Replies
Login or Register to Ask a Question