Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to remove FIRST Line of huge text file on Solaris Post 302122324 by porter on Wednesday 20th of June 2007 04:42:12 AM
Old 06-20-2007
Personally I would write a small C program that would

1. find the length of the first line

2. do repetative read/writes to shuffle the data up

3. use ftruncate to remove the length from the end of the file

However I would also recommend you back this file up.

If you know how many lines there actually are....

Code:
wc -l <bigfile.txt

then if there are 70, do the following...

Code:
tail -69 <bigfile.txt >bigfile2.txt

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies

2. Shell Programming and Scripting

sed to remove 1st two characters every line of text file

what is the sed command to remove the first two characters of every line of a text file? each line of the text file has the same amount of characters, and they are ALL NUMERIC. there are hundreds of lines though. for example, >cat file1.txt 10081551 10081599 10082234 10082259 20081134... (20 Replies)
Discussion started by: ajp7701
20 Replies

3. Shell Programming and Scripting

Replacing in huge text file

I have huge text files (~120 MB)x100 which equivalents to ~11GB of data. The files contain pure numbers, actually the value of "phi" to 10 billion digits!! I know its huge!! Here are the last few lines of a file 0952899155 3233967444 3344925499 0276061529 7261968933 9683989044 3317145063... (14 Replies)
Discussion started by: shantanuthatte
14 Replies

4. Shell Programming and Scripting

How to open a huge text file?

Hi. I have a 10 Gb text file.the default text editor in ubuntu doens't open it. Does anyone know how can i open it?? Thanks (4 Replies)
Discussion started by: stalaei
4 Replies

5. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
19 Replies

6. Shell Programming and Scripting

Output only first 400 bytes of a huge text file

How do I output only the first 400 bytes of a huge text file to a new file. It has to be unmodified so no added invisible characters. Many thanks..... (3 Replies)
Discussion started by: garethsays
3 Replies

7. Shell Programming and Scripting

Edit a Huge one line file

We have a huge file which has just one really large line; about 500 MB. I want to 1. Count all the occurrences of a phrase 2. Replace the phrase with another. Trying to open it using vi has not helped as it complains that it is too large. Can any script help? Please advise. Thank you, (12 Replies)
Discussion started by: kaushikadya
12 Replies

8. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

9. Shell Programming and Scripting

Reading ALL BUT the first and last line of a huge file

Hi. Pardon me if I'm posting a duplicate thread but.. I have a text file with over 150 Million records, file size is in the range if MB(close to GB). The requirement is to read ALL the lines excepting the FIRST LINE which is the file header and the LAST LINE which is it's trailer record. ... (8 Replies)
Discussion started by: kumarjt
8 Replies

10. UNIX for Beginners Questions & Answers

Tip to remove line endings and spaces on a pre-formatted text file?

Hi, At the moment, using Notepad++ to do a search and replace, manually section by section which is real painful. Yeah, so copying each section of the line of text and putting into a file and then search and replace, need at least 3-operations in Notepad++. Here's hoping I will be able to... (1 Reply)
Discussion started by: newbie_01
1 Replies
TRUNCATE(2)						      BSD System Calls Manual						       TRUNCATE(2)

NAME
ftruncate, truncate -- truncate or extend a file to a specified length SYNOPSIS
#include <unistd.h> int ftruncate(int fildes, off_t length); int truncate(const char *path, off_t length); DESCRIPTION
ftruncate() and truncate() cause the file named by path, or referenced by fildes, to be truncated (or extended) to length bytes in size. If the file size exceeds length, any extra data is discarded. If the file size is smaller than length, the file is extended and filled with zeros to the indicated length. The ftruncate() form requires the file to be open for writing. Note: ftruncate() and truncate() do not modify the current file offset for any open file descriptions associated with the file. RETURN VALUES
A value of 0 is returned if the call succeeds. If the call fails a -1 is returned, and the global variable errno specifies the error. ERRORS
The ftruncate() system call will fail if: [EBADF] fildes is not a valid descriptor open for writing. [EFBIG] The file is a regular file and length is greater than the offset maximum established in the open file description associ- ated with fildes. [EINVAL] fildes references a socket, not a file. [EINVAL] fildes is not open for writing. [EROFS] The named file resides on a read-only file system. The truncate() system call will fail if: [EACCES] Search permission is denied for a component of the path prefix. [EACCES] The named file is not writable by the user. [EFAULT] Path points outside the process's allocated address space. [EISDIR] The named file is a directory. [ELOOP] Too many symbolic links are encountered in translating the pathname. This is taken to be indicative of a looping symbolic link. [ENAMETOOLONG] A component of a pathname exceeds {NAME_MAX} characters, or an entire path name exceeds {PATH_MAX} characters. [ENOENT] The named file does not exist. [ENOTDIR] A component of the path prefix is not a directory. [EROFS] The named file resides on a read-only file system. [ETXTBSY] The file is a pure procedure (shared text) file that is being executed. The ftruncate() and truncate() system calls will fail if: [EFBIG] The length argument was greater than the maximum file size. [EINTR] A signal is caught during execution. [EINVAL] The length argument is less than 0. [EIO] An I/O error occurred while reading from or writing to a file system. SEE ALSO
open(2) BUGS
These calls should be generalized to allow ranges of bytes in a file to be discarded. Use of truncate() to extend a file is not portable. HISTORY
The truncate() and ftruncate() function calls appeared in 4.2BSD. 4.2 Berkeley Distribution June 4, 1993 4.2 Berkeley Distribution
All times are GMT -4. The time now is 03:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy