Sponsored Content
Top Forums Shell Programming and Scripting Help on splitting this huge file Post 302325240 by turk451 on Saturday 13th of June 2009 10:36:59 PM
Old 06-13-2009
You might want to see if the tail command would work for your purposes. I see that it has a byte option, so you could most likely do something like:

tail -c 700 [file]

Just check the man entry for tail on your system to see that it supports this option.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting huge XML Files into fixsized wellformed parts

Hi, I need to split xml-files with sizes greater than 2 gb into smaler chunks. As I dont want to end up with billions of files, I want those splitted files to have configurable sizes like 250 MB. Each file should be well formed having an exact copy of the header (and footer as the closing of the... (0 Replies)
Discussion started by: Malapha
0 Replies

2. Shell Programming and Scripting

sorting huge file

Hi All I am sorting a huge file -rw-r--r-- 1 rama users 448156978 May 13 18:48 102384.temp $ sort -k 1,40n 102384.temp > 102384.temp1 msgcnt 1468 vxfs: mesg 001: vx_nospace - /dev/vg00/var file system full (1 block extent) sort: A write error occurred while sorting. I thought... (3 Replies)
Discussion started by: dhanamurthy
3 Replies

3. Shell Programming and Scripting

splitting huge xml into multiple files

hi all i have a some huge html files (500MB to 1GB). Each file has multiple <html></html> tags <html> ................. .................... .................... </html> <html> ................. .................... .................... </html> <html> .................... (5 Replies)
Discussion started by: uttamhoode
5 Replies

4. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

5. Shell Programming and Scripting

Huge File Comparison

Hi i need to compare two fixed length files and produce the differences if any to a seperate file. I have to capture each and every differneces line by line. Ideally my files should not have any differences but if there are any then it should be captured without any miss. Also my files sizes are... (4 Replies)
Discussion started by: naveenn08
4 Replies

6. Shell Programming and Scripting

Splitting the Huge file into several files...

Hi I have to write a script to split the huge file into several pieces. The file columns is | pipe delimited. The data sample is as: 6625060|1420215|07308806|N|20100120|5572477081|+0002.79|+0000.00|0004|0001|......... (3 Replies)
Discussion started by: lakteja
3 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

9. Shell Programming and Scripting

Need help splitting huge single record file

I was given a data file that I need to split into multiple lines/records based on a key word. The problem is that it is 2.5GB or bigger and everything I try in perl or sed causes a Segmentation fault. Can someone give me some other ideas. The data is of the form:... (5 Replies)
Discussion started by: leolson
5 Replies

10. UNIX for Dummies Questions & Answers

My file system is 100%, can't find the huge file

Please help. My file system is 100%, I can't seem to find what is taking so much space. The total hard drive space is 150Gig free but I got nothing now. I did to this to find the big file but it's taking so much time. Is there any other way? du -ah / | more find ./ -size +200M... (3 Replies)
Discussion started by: samnyc
3 Replies
tail(1) 																   tail(1)

NAME
tail - deliver the last part of a file SYNOPSIS
/usr/bin/tail [ +-s number [lbcr]] [file] /usr/bin/tail [-lbcr] [file] /usr/bin/tail [ +- number [lbcf]] [file] /usr/bin/tail [-lbcf] [file] /usr/xpg4/bin/tail [-f | -r] [-c number | -n number] [file] /usr/xpg4/bin/tail [ +- number [l | b | c] [f]] [file] /usr/xpg4/bin/tail [ +- number [l] [f | r] ] [file] The tail utility copies the named file to the standard output beginning at a designated place. If no file is named, the standard input is used. Copying begins at a point in the file indicated by the -cnumber, -nnumber, or +-number options (if +number is specified, begins at distance number from the beginning; if -number is specified, from the end of the input; if number is NULL, the value 10 is assumed). number is counted in units of lines or byte according to the -c or -n options, or lines, blocks, or bytes, according to the appended option l, b, or c. When no units are specified, counting is by lines. The following options are supported for both /usr/bin/tail and /usr/xpg4/bin/tail. The -r and -f options are mutually exclusive. If both are specified on the command line, the -f option is ignored. -b Units of blocks. -c Units of bytes. -f Follow. If the input-file is not a pipe, the program does not terminate after the line of the input-file has been copied, but enters an endless loop, wherein it sleeps for a second and then attempts to read and copy further records from the input-file. Thus it can be used to monitor the growth of a file that is being written by some other process. -l Units of lines. -r Reverse. Copies lines from the specified starting point in the file in reverse order. The default for r is to print the entire file in reverse order. /usr/xpg4/bin/tail The following options are supported for /usr/xpg4/bin/tail only: -c number The number option-argument must be a decimal integer whose sign affects the location in the file, measured in bytes, to begin the copying: + Copying starts relative to the beginning of the file. - Copying starts relative to the end of the file. none Copying starts relative to the end of the file. The origin for counting is 1; that is, -c+1 represents the first byte of the file, -c-1 the last. -n number Equivalent to -cnumber, except the starting location in the file is measured in lines instead of bytes. The origin for counting is 1. That is, -n+1 represents the first line of the file, -n-1 the last. The following operand is supported: file A path name of an input file. If no file operands are specified, the standard input is used. See largefile(5) for the description of the behavior of tail when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). Example 1: Using the tail Command The following command prints the last ten lines of the file fred, followed by any lines that are appended to fred between the time tail is initiated and killed. example% tail -f fred The next command prints the last 15 bytes of the file fred, followed by any lines that are appended to fred between the time tail is initi- ated and killed: example% tail -15cf fred See environ(5) for descriptions of the following environment variables that affect the execution of tail: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. The following exit values are returned: 0 Successful completion. >0 An error occurred. See attributes(5) for descriptions of the following attributes: /usr/bin/tail +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ /usr/xpg4/bin/tail +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWxcu4 | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ cat(1), head(1), more(1), pg(1), dd(1M), attributes(5), environ(5), largefile(5), standards(5) Piped tails relative to the end of the file are stored in a buffer, and thus are limited in length. Various kinds of anomalous behavior can happen with character special files. 13 Jul 2005 tail(1)
All times are GMT -4. The time now is 04:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy