Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to remove FIRST Line of huge text file on Solaris Post 302122324 by porter on Wednesday 20th of June 2007 04:42:12 AM
Old 06-20-2007
Personally I would write a small C program that would

1. find the length of the first line

2. do repetative read/writes to shuffle the data up

3. use ftruncate to remove the length from the end of the file

However I would also recommend you back this file up.

If you know how many lines there actually are....

Code:
wc -l <bigfile.txt

then if there are 70, do the following...

Code:
tail -69 <bigfile.txt >bigfile2.txt

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. Shell Programming and Scripting

sed to remove 1st two characters every line of text file

what is the sed command to remove the first two characters of every line of a text file? each line of the text file has the same amount of characters, and they are ALL NUMERIC. there are hundreds of lines though. for example, >cat file1.txt 10081551 10081599 10082234 10082259 20081134... (20 Replies)
Discussion started by: ajp7701
20 Replies

3. Shell Programming and Scripting

Replacing in huge text file

I have huge text files (~120 MB)x100 which equivalents to ~11GB of data. The files contain pure numbers, actually the value of "phi" to 10 billion digits!! I know its huge!! Here are the last few lines of a file 0952899155 3233967444 3344925499 0276061529 7261968933 9683989044 3317145063... (14 Replies)
Discussion started by: shantanuthatte
14 Replies

4. Shell Programming and Scripting

How to open a huge text file?

Hi. I have a 10 Gb text file.the default text editor in ubuntu doens't open it. Does anyone know how can i open it?? Thanks (4 Replies)
Discussion started by: stalaei
4 Replies

5. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
19 Replies

6. Shell Programming and Scripting

Output only first 400 bytes of a huge text file

How do I output only the first 400 bytes of a huge text file to a new file. It has to be unmodified so no added invisible characters. Many thanks..... (3 Replies)
Discussion started by: garethsays
3 Replies

7. Shell Programming and Scripting

Edit a Huge one line file

We have a huge file which has just one really large line; about 500 MB. I want to 1. Count all the occurrences of a phrase 2. Replace the phrase with another. Trying to open it using vi has not helped as it complains that it is too large. Can any script help? Please advise. Thank you, (12 Replies)
Discussion started by: kaushikadya
12 Replies

8. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

9. Shell Programming and Scripting

Reading ALL BUT the first and last line of a huge file

Hi. Pardon me if I'm posting a duplicate thread but.. I have a text file with over 150 Million records, file size is in the range if MB(close to GB). The requirement is to read ALL the lines excepting the FIRST LINE which is the file header and the LAST LINE which is it's trailer record. ... (8 Replies)
Discussion started by: kumarjt
8 Replies

10. UNIX for Beginners Questions & Answers

Tip to remove line endings and spaces on a pre-formatted text file?

Hi, At the moment, using Notepad++ to do a search and replace, manually section by section which is real painful. Yeah, so copying each section of the line of text and putting into a file and then search and replace, need at least 3-operations in Notepad++. Here's hoping I will be able to... (1 Reply)
Discussion started by: newbie_01
1 Replies
FTRUNCATE(P)						     POSIX Programmer's Manual						      FTRUNCATE(P)

NAME
ftruncate - truncate a file to a specified length SYNOPSIS
#include <unistd.h> int ftruncate(int fildes, off_t length); DESCRIPTION
If fildes is not a valid file descriptor open for writing, the ftruncate() function shall fail. If fildes refers to a regular file, the ftruncate() function shall cause the size of the file to be truncated to length. If the size of the file previously exceeded length, the extra data shall no longer be available to reads on the file. If the file previously was smaller than this size, ftruncate() shall either increase the size of the file or fail. XSI-conformant systems shall increase the size of the file. If the file size is increased, the extended area shall appear as if it were zero-filled. The value of the seek pointer shall not be modi- fied by a call to ftruncate(). Upon successful completion, if fildes refers to a regular file, the ftruncate() function shall mark for update the st_ctime and st_mtime fields of the file and the S_ISUID and S_ISGID bits of the file mode may be cleared. If the ftruncate() function is unsuccessful, the file is unaffected. If the request would cause the file size to exceed the soft file size limit for the process, the request shall fail and the implementation shall generate the SIGXFSZ signal for the thread. If fildes refers to a directory, ftruncate() shall fail. If fildes refers to any other file type, except a shared memory object, the result is unspecified. If fildes refers to a shared memory object, ftruncate() shall set the size of the shared memory object to length. If the effect of ftruncate() is to decrease the size of a shared memory object or memory mapped file and whole pages beyond the new end were previously mapped, then the whole pages beyond the new end shall be discarded. If the Memory Protection option is supported, references to discarded pages shall result in the generation of a SIGBUS signal; otherwise, the result of such references is undefined. If the effect of ftruncate() is to increase the size of a shared memory object, it is unspecified whether the contents of any mapped pages between the old end-of-file and the new are flushed to the underlying object. RETURN VALUE
Upon successful completion, ftruncate() shall return 0; otherwise, -1 shall be returned and errno set to indicate the error. ERRORS
The ftruncate() function shall fail if: EINTR A signal was caught during execution. EINVAL The length argument was less than 0. EFBIG or EINVAL The length argument was greater than the maximum file size. EFBIG The file is a regular file and length is greater than the offset maximum established in the open file description associated with fildes. EIO An I/O error occurred while reading from or writing to a file system. EBADF or EINVAL The fildes argument is not a file descriptor open for writing. EINVAL The fildes argument references a file that was opened without write permission. EROFS The named file resides on a read-only file system. The following sections are informative. EXAMPLES
None. APPLICATION USAGE
None. RATIONALE
The ftruncate() function is part of IEEE Std 1003.1-2001 as it was deemed to be more useful than truncate(). The truncate() function is provided as an XSI extension. FUTURE DIRECTIONS
None. SEE ALSO
open() , truncate() , the Base Definitions volume of IEEE Std 1003.1-2001, <unistd.h> COPYRIGHT
Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol- ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html . IEEE
/The Open Group 2003 FTRUNCATE(P)
All times are GMT -4. The time now is 01:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy