Hi all,
I am working on an extremely large collection of text data (about 2 million XML files) in a directory. I have changed the extension from .xml to .dat. Right now I am using this code to remove the XML tags, but the code is way too slow. It seems that it is taking fore-ever:
Just to let the readers know that the commented line with ls does not even work as it gives Argument list too long message.
Then I modified the code, and came up with this:
Just wish to know will this speed up my task? What I want to do is that instead of doing ls or find, I should generate the filename using my code, and the program should then process that file which has been automatically generated. The trick that I have used is that I have re-named all the 2 million files with "contiguous" numbers 1.dat, 2.dat, 3.dat, 4.dat and so on without leaving any number in between and then using a counter, I generate those numbers and read those files.
Or, Is there any other better way to fasten up my task? I am using Linux with BASH.
No matter which way you do it, you're still running lynx 2 million times. Which do you think is the holdup -- the tiny shell loop, or the part which does all the actual work?
Last edited by Corona688; 07-09-2013 at 12:46 PM..
Hi Bigshots,
I have a pattern file with two columns. I have another data file. If column 1 in the pattern file appears as the 4th column in the data file, I need to replace it (4th column of data file) with column 2 of the pattern file. If the pattern is found in any other column, it should not... (6 Replies)
Hello,
I am trying to colour code a single word in whole line. Can you please help.
I am able to colour code the whole line but not able to do only for single word
Query) I want to echo below line and colur code red to word FAILED only.
This server is FAILED in check. (2 Replies)
hi,
i want to pop up an alert box using perl script. my requirement is.
i am using a html page which calls a perl script. this perl script calls a shell script.. after the shell script ends its execution, i am using exit 0 to terminate the shell script successfully and exit 1 to terminate the... (3 Replies)
Hello all,
I am in a middle of an assignment and i would appreciate any help.
How can i write a bash shell script code that checks if all elements in an array are the same numbers. I mean -->array = ( 0,0,0,0,0 )
( e.g., if
then return "OK'
fi )
Thank you in advance, (9 Replies)
Hello,
I try to run a script shell from a java program:
but it runs only if i do :chmod 777 myShellScript in the terminal
Please how can i insert chmod 777 in my java code without going through the terminal?
Thank you (1 Reply)
Hi there,
I had run into some fortran code to modify. Obviously, it was written without thinking of high performance computing and not parallelized... Now I would like to make the code "on track" and parallel. After a whole afternoon thinking, I still cannot find where to start. Can any one... (3 Replies)
Hello,
Error
awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt
What it is
It is a unix shell script that contains an awk program as well as... (4 Replies)
Hi, I have a very large, very old FORTRAN code that I work with. The code is quite messy and I was wondering if I can speed up execution time by finding subroutines that code execution spends the most time in. Is there any kind of software I can use to see where the code spends most of the... (1 Reply)
hello,
do anybody know a program to format a shell script code ?
i tried "editrocket.com" but this product doesn't format a shell script code.
i searched for programs but can't find a shell script code formatter.
i have to change a shell script and the style of code is .....
regards (5 Replies)