How to remove FIRST Line of huge text file on Solaris


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to remove FIRST Line of huge text file on Solaris
# 1  
Old 06-20-2007
Error How to remove FIRST Line of huge text file on Solaris

i need help..!!!!

i have one big text file estimate data file size 50 - 100GB with 70 Mega Rows.
on OS SUN Solaris version 8

How i can remove first line of the text file.

Please suggest me for solutions.

Thank you very much in advanceSmilie
# 2  
Old 06-20-2007
Data

i tried to use :

$ head -1 bigfile.txt > bigfile2.txt
$

But line 1 of bigfile.txt don't removed from bigfile.txt. It's just copy first line from bigfile.txt to bigfile2.txt Only.

T_T
# 3  
Old 06-20-2007
Personally I would write a small C program that would

1. find the length of the first line

2. do repetative read/writes to shuffle the data up

3. use ftruncate to remove the length from the end of the file

However I would also recommend you back this file up.

If you know how many lines there actually are....

Code:
wc -l <bigfile.txt

then if there are 70, do the following...

Code:
tail -69 <bigfile.txt >bigfile2.txt

# 4  
Old 06-20-2007
I would try:
sed 1d < bigfile.txt >bigfile2.txt
# 5  
Old 06-20-2007
MySQL thk

Thank you very much krab. ^^

i can solved it ^^ Smilie
# 6  
Old 06-23-2007
Quote:
Originally Posted by madoatz
i need help..!!!!

i have one big text file estimate data file size 50 - 100GB with 70 Mega Rows.
on OS SUN Solaris version 8

How i can remove first line of the text file.

Please suggest me for solutions.

Thank you very much in advanceSmilie
If you know the pattern of first line or some word which does not repeat else where in the file,then you can use grep command.

cat filename | grep -v 'pattern' > file_temp
mv file_temp > filename
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Tip to remove line endings and spaces on a pre-formatted text file?

Hi, At the moment, using Notepad++ to do a search and replace, manually section by section which is real painful. Yeah, so copying each section of the line of text and putting into a file and then search and replace, need at least 3-operations in Notepad++. Here's hoping I will be able to... (1 Reply)
Discussion started by: newbie_01
1 Replies

2. Shell Programming and Scripting

Reading ALL BUT the first and last line of a huge file

Hi. Pardon me if I'm posting a duplicate thread but.. I have a text file with over 150 Million records, file size is in the range if MB(close to GB). The requirement is to read ALL the lines excepting the FIRST LINE which is the file header and the LAST LINE which is it's trailer record. ... (8 Replies)
Discussion started by: kumarjt
8 Replies

3. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

4. Shell Programming and Scripting

Edit a Huge one line file

We have a huge file which has just one really large line; about 500 MB. I want to 1. Count all the occurrences of a phrase 2. Replace the phrase with another. Trying to open it using vi has not helped as it complains that it is too large. Can any script help? Please advise. Thank you, (12 Replies)
Discussion started by: kaushikadya
12 Replies

5. Shell Programming and Scripting

Output only first 400 bytes of a huge text file

How do I output only the first 400 bytes of a huge text file to a new file. It has to be unmodified so no added invisible characters. Many thanks..... (3 Replies)
Discussion started by: garethsays
3 Replies

6. Shell Programming and Scripting

How to fix line breaks format text for huge files?

Hi, I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly. Except the header and trailer, each record starts with 'D'. Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
19 Replies

7. Shell Programming and Scripting

How to open a huge text file?

Hi. I have a 10 Gb text file.the default text editor in ubuntu doens't open it. Does anyone know how can i open it?? Thanks (4 Replies)
Discussion started by: stalaei
4 Replies

8. Shell Programming and Scripting

Replacing in huge text file

I have huge text files (~120 MB)x100 which equivalents to ~11GB of data. The files contain pure numbers, actually the value of "phi" to 10 billion digits!! I know its huge!! Here are the last few lines of a file 0952899155 3233967444 3344925499 0276061529 7261968933 9683989044 3317145063... (14 Replies)
Discussion started by: shantanuthatte
14 Replies

9. Shell Programming and Scripting

sed to remove 1st two characters every line of text file

what is the sed command to remove the first two characters of every line of a text file? each line of the text file has the same amount of characters, and they are ALL NUMERIC. there are hundreds of lines though. for example, >cat file1.txt 10081551 10081599 10082234 10082259 20081134... (20 Replies)
Discussion started by: ajp7701
20 Replies

10. UNIX for Advanced & Expert Users

Insert a line as the first line into a very huge file

Hello, I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it? Any help would be... (3 Replies)
Discussion started by: shriek
3 Replies
Login or Register to Ask a Question