11-29-2007
This link shows how to split/recombine any file, into whatever size you want:
fsplit -- splitting up a binary file
However, an excel file would need to be recombined to work correctly, as its formatted in such a way, i.e, a binary file.you can't just cut it up and use the pieces.
I've used this to split >2G binary files, to put on fileservers that are not largefile aware, then they get copied to thier destination, recombined, and
everything works just fine.
The real fix for your delima will be in changing the program thats creating your "to large" excel file, to having a limit of ( 64k - some safety factor ), and putting the remainder into another file. Programs/scripts that create any sort of data, files, etc really need to be aware of the limitation of whatever will be using output it creates.
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
I have a large file with the first 2 characters of each line determining the type of record. type 03 being a subheader and then it will have multiple 04 records.
eg: 03,xxx,xxxx,xxxx
04,xxxxxxxxxxxxxxxxxxxxxxxxxxxx
04,xxxxxxxxxxxxxxxxxxxxxxxxxxxx
03,xxx,xxx,xxx
... (2 Replies)
Discussion started by: badg3r
2 Replies
2. UNIX for Dummies Questions & Answers
I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this.
For example:
split -l 3000000 filename.txt
This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies
3. Shell Programming and Scripting
Hi All,
I have a strange behaviour from csplit command on some text files.
I have a comma separated file and data with in double quotes. This file is being generated from Sequential stage of Data stage tool.
This file has 67 fields in each records with some null and blank in the data.... (4 Replies)
Discussion started by: shreekrishnagd
4 Replies
4. UNIX for Dummies Questions & Answers
Hi,
This is my first time on this forum..
I searched the previous answers, but didn't find the answer I was looking for at first glance.
csplit works beautifully for me, except for one thing. My file looks like this:
ad|name1|asdf...(several pages)..asdf ...
ad|name2|asdf...(several... (8 Replies)
Discussion started by: juliette salexa
8 Replies
5. Shell Programming and Scripting
Hello all
The below command works perfeft when executed from the shell prompt.
"csplit -f first Allocation.log.1 "%. Oct 02 .%" {1} "
and generates first00 file.
But it fails if I include the same command in my script.
I am trying to automate some process and this is turning... (1 Reply)
Discussion started by: uandme2k2
1 Replies
6. Shell Programming and Scripting
I have a file with contents
<wmqi>
sdf
sdf
sdffghghhjjfh
</wmqi>
<wmqi>
gh
dfg
hhjhj
sdfsdf
g
</wmqi>
<wmqi>
dfgdf
fg
dfgfg
</wmqi>
<wmqi> (6 Replies)
Discussion started by: Shivdatta
6 Replies
7. UNIX for Dummies Questions & Answers
I want to split big files based on a pattern to stdout.
Although csplit works well for me splitting the output into separate files (e.g. xx00, xx01, xx02, ...), the following is not working as expected:
<code>
# assuming pattern occurs less than 100 times
csplit bigfile '%pattern%'... (2 Replies)
Discussion started by: uiop44
2 Replies
8. Shell Programming and Scripting
Hello,
I want to split a big file into smaller ones with certain "counts". I am aware this type of job has been asked quite often, but I posted again when I came to csplit, which may be simpler to solve the problem.
Input file (fasta format):
>seq1
agtcagtc
agtcagtc
ag
>seq2
agtcagtcagtc... (8 Replies)
Discussion started by: yifangt
8 Replies
9. UNIX for Beginners Questions & Answers
Hello,
I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number.
What I have tried is the below command with 2 digit numeric value
split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02)
How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies
10. UNIX for Beginners Questions & Answers
i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies
LEARN ABOUT DEBIAN
largefile
largefile(5) Standards, Environments, and Macros largefile(5)
NAME
largefile - large file status of utilities
DESCRIPTION
A large file is a regular file whose size is greater than or equal to 2 Gbyte ( 2**31 bytes). A small file is a regular file whose size is
less than 2 Gbyte.
Large file aware utilities
A utility is called large file aware if it can process large files in the same manner as it does small files. A utility that is large file
aware is able to handle large files as input and generate as output large files that are being processed. The exception is where additional
files are used as system configuration files or support files that can augment the processing. For example, the file utility supports the
-m option for an alternative "magic" file and the -f option for a support file that can contain a list of file names. It is unspecified
whether a utility that is large file aware will accept configuration or support files that are large files. If a large file aware utility
does not accept configuration or support files that are large files, it will cause no data loss or corruption upon encountering such files
and will return an appropriate error.
The following /usr/bin utilities are large file aware:
adb awk bdiff cat chgrp
chmod chown cksum cmp compress
cp csh csplit cut dd
dircmp du egrep fgrep file
find ftp getconf grep gzip
head join jsh ksh ln
ls mdb mkdir mkfifo more
mv nawk page paste pathchck
pg rcp remsh rksh rm
rmdir rsh sed sh sort
split sum tail tar tee
test touch tr uncompress uudecode
uuencode wc zcat
The following /usr/xpg4/bin utilities are large file aware:
awk cp chgrp chown du
egrep fgrep file grep ln
ls more mv rm sed
sh sort tail tr
The following /usr/xpg6/bin utilities are large file aware:
getconf ls tr
The following /usr/sbin utilities are large file aware:
install mkfile mknod mvdir swap
See the USAGE section of the swap(1M) manual page for limitations of swap on block devices greater than 2 Gbyte on a 32-bit operating sys-
tem.
The following /usr/ucb utilities are large file aware:
chown from ln ls sed
sum touch
The /usr/bin/cpio and /usr/bin/pax utilities are large file aware, but cannot archive a file whose size exceeds 8 Gbyte - 1 byte.
The /usr/bin/truss utilities has been modified to read a dump file and display information relevant to large files, such as offsets.
cachefs file systems
The following /usr/bin utilities are large file aware for cachefs file systems:
cachefspack cachefsstat
The following /usr/sbin utilities are large file aware for cachefs file systems:
cachefslog cachefswssize cfsadmin fsck
mount umount
nfs file systems
The following utilities are large file aware for nfs file systems:
/usr/lib/autofs/automountd /usr/sbin/mount
/usr/lib/nfs/rquotad
ufs file systems
The following /usr/bin utility is large file aware for ufs file systems:
df
The following /usr/lib/nfs utility is large file aware for ufs file systems:
rquotad
The following /usr/xpg4/bin utility is large file aware for ufs file systems:
df
The following /usr/sbin utilities are large file aware for ufs file systems:
clri dcopy edquota ff fsck
fsdb fsirand fstyp labelit lockfs
mkfs mount ncheck newfs quot
quota quotacheck quotaoff quotaon repquota
tunefs ufsdump ufsrestore umount
Large file safe utilities
A utility is called large file safe if it causes no data loss or corruption when it encounters a large file. A utility that is large file
safe is unable to process properly a large file, but returns an appropriate error.
The following /usr/bin utilities are large file safe:
audioconvert audioplay audiorecord comm diff
diff3 diffmk ed lp mail
mailcompat mailstats mailx pack pcat
red rmail sdiff unpack vi
view
The following /usr/xpg4/bin utilities are large file safe:
ed vi view
The following /usr/xpg6/bin utility is large file safe:
ed
The following /usr/sbin utilities are large file safe:
lpfilter lpforms
The following /usr/ucb utilities are large file safe:
Mail lpr
The following /usr/lib utility is large file safe:
sendmail
SEE ALSO
lf64(5), lfcompile(5), lfcompile64(5)
SunOS 5.10 7 Nov 2003 largefile(5)