10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Pardon me if I'm posting a duplicate thread but..
I have a text file with over 150 Million records, file size is in the range if MB(close to GB).
The requirement is to read ALL the lines excepting the FIRST LINE which is the file header and the LAST LINE which is it's trailer record.
... (8 Replies)
Discussion started by: kumarjt
2. UNIX for Dummies Questions & Answers
I would like to replace new line characters(\n) in a huge file of about 2 million records . I tried this one (:%s/\n//g) but it's hanging there and no result. Does this command do not work if the file is big. Please let me know if you have any other options
Raj (1 Reply)
Discussion started by: rajeevm
3. Shell Programming and Scripting
We have a huge file which has just one really large line; about 500 MB. I want to
1. Count all the occurrences of a phrase
2. Replace the phrase with another.
Trying to open it using vi has not helped as it complains that it is too large. Can any script help? Please advise.
Thank you, (12 Replies)
Discussion started by: kaushikadya
4. Shell Programming and Scripting
I need to correct line breaks for huge files (more than 1MM records in a file) and then format it properly.
Except the header and trailer, each record starts with 'D'.
Requirement:Scan the whole file except the header and trailer records and see if any of the records start with... (19 Replies)
Discussion started by: kikionline
5. Shell Programming and Scripting
I have huge text files (~120 MB)x100 which equivalents to ~11GB of data. The files contain pure numbers, actually the value of "phi" to 10 billion digits!!
I know its huge!! Here are the last few lines of a file
0952899155 3233967444 3344925499 0276061529 7261968933 9683989044 3317145063... (14 Replies)
Discussion started by: shantanuthatte
6. Shell Programming and Scripting
I am having 2 huge files having line count more than 10million. The files look like:
44415539519 TQ03... (2 Replies)
Discussion started by: rochitsharma
7. Shell Programming and Scripting
I've got two folders, say, "folder1" and "folder2".
Under each, there are thousands of files.
It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command.
However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
8. Shell Programming and Scripting
I got a file named as data_file
Contents of data_file got about IGB reads.
The contents look like below:
WEEGFVShhhhhhhhhhhhPP... (7 Replies)
Discussion started by: patrick87
9. UNIX for Dummies Questions & Answers
i need help..!!!!
i have one big text file estimate data file size 50 - 100GB with 70 Mega Rows.
on OS SUN Solaris version 8
How i can remove first line of the text file.
Please suggest me for solutions.
Thank you very much in advance:) (5 Replies)
Discussion started by: madoatz
10. UNIX for Advanced & Expert Users
I need to insert a line (like a header) as the first line of a very huge file (about 3 ml rows). I am able to do it with sed, but redirecting the output and creating a new file takes quite some time. I was wondering if there was a more efficient way of doing it?
Any help would be... (3 Replies)
Discussion started by: shriek