10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have big files (some are >300GB!) that need substitution for some patterns, for example, change Multiple Spaces into Tab. I used this oneliner:sed '1,18s/ \{1,\}/\t/g' infile_big.sam > outfile_big.sambut it seems very slow as the job is still running after 24 hours! In this example, only the... (8 Replies)
Discussion started by: yifangt
8 Replies
2. Shell Programming and Scripting
Hi,
I have a huge file with a single line.
But I want to break that line into lines of with each line having five columns.
My file is like this:
code:
"hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you."
I want it like this:
code:... (1 Reply)
Discussion started by: rajsharma
1 Replies
3. Shell Programming and Scripting
into small files. i need to add a head.txt and tail.txt into small files at the begin and end, and give a name as q1.xml q2.xml q3.xml ....
thank you very much. (2 Replies)
Discussion started by: dtdt
2 Replies
4. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
5. Shell Programming and Scripting
Hi,
I have one file, say file 1, that has data like below where 19900107 is the date,
19900107 12 144 129 0.7380047
19900108 12 168 129 0.3149017
19900109 12 192 129 3.2766666E-02
... (3 Replies)
Discussion started by: Wynner
3 Replies
6. Shell Programming and Scripting
Hi,
I would like to clarify about the NAWK array to store multiple lines from huge file.
The file is having an unique REF.NO, I wants to store the lines (it may be
100+ lines) till I found the new REF.NO.
How can I apply NAWK - arrays for the above?
Rgds,
sharif. (1 Reply)
Discussion started by: sharif
1 Replies
7. UNIX for Dummies Questions & Answers
Dear All,
I am working with windoes OS but remote a linux machine. I wonder the way to copy an paste some part of a huge file in linux machine.
the contain of file like as follow:
...
dump annealling all custom 10 anneal_*.dat id type x y z q
timestep 0.02
run 200000
Memory... (2 Replies)
Discussion started by: ariesto
2 Replies
8. Shell Programming and Scripting
>cat data.dat
0001 Robbert
0002 Nick
0003 Mark
.......
1000 Jarek (3 Replies)
Discussion started by: McLan
3 Replies
9. Shell Programming and Scripting
I have to delete 1st 7000 lines of a file which is 12GB large. As it is so large, i can't open in vi and delete these lines. Also I found one post here which gave solution using perl, but I don't have perl installed. Also some solutions were redirecting the o/p to a different file and renaming it.... (3 Replies)
Discussion started by: rahulrathod
3 Replies
10. Shell Programming and Scripting
I am reading a file that contains over 5000 lines and I want to assign it to a shell variable array (which has a restriction of 1024 rows). I had an idea that if I could grab 1000 record hunks of the file, and pipe the records out, that I could perform a loop until I got to the end and process 1000... (5 Replies)
Discussion started by: beilstwh
5 Replies