I'm reading in some >41,000 line files and doing some manipulations of columns based on the values of other columns. Arrays make a ton of sense for this application.
Not to slow it down too much, I want to spit out the lines efficiently and not have another loop indexing the entries for each line.
For each row
Code:
$echo "${array[@]}
gives space delimiters (and some of the values contain space so piping it through tr won't work)
Code:
IFS=','; echo "${array[@]}
does not change anything, the output is still space delimited (I was not expecting this to work).
Is there time and processor power efficient way to do this without a loop indexing through all the columns?
Mike
PS. topic related to the code
Code:
IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]} >> out_file
done < in_file
is much much slower than
Code:
(IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]}
done ) < in_file > out_file
I suspect this is because the file is only opened and closed once, not 41,000 times. What are the risks of the subshell over-running it's available memory? I'm looking at ~10 mb files for just Q1 2013 data and I still need to port over data from 2010 on. This will have to be deployed to a Cygwin enviroment.
Hello.
System : opensuse leap 42.3
I have a bash script that build a text file.
I would like the last command doing :
print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt
where :
print_cmd ::= some printing... (1 Reply)
I came across and unexpected behavior with redirections in tcsh. I know, csh is not best for redirections, but I'd like to understand what is happening here.
I have following script (called out_to_streams.csh):
#!/bin/tcsh -f
echo Redirected to STDOUT > /dev/stdout
echo Redirected to... (2 Replies)
Hi,
I'm dealing with an issue and losing a lot of hours figuring out how i would solve this.
I have an input file which looks like this:
('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos')
('BLABLA +200-GRR','Serviço... (6 Replies)
Hi Folks,
As per the subject, the following command is not working as expected.
echo $variable | mail -s "subject" "xxx@xxx.com"
Could anyone figure it out whats wrong with this. I am using AIX box.
Regards, (2 Replies)
Hi,
I have line in input file as below:
3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL
My expected output for line in the file must be :
"1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL"
Can someone... (7 Replies)
echo `echo ` doesn't echoes anything. And it's logic. But
echo `echo `echo ` ` does echoes "echo". What's the logic of it? the `echo `echo ` inside of the whole (first) echo, echoes nothing, so the first echo have to echo nothing but echoes "echo"
(too much echoing :P):o (2 Replies)
Hello,
I am working on solving an NP-Complete problem, so it is very important that operations and data with limited integer-argument ranges be computed using immutable look-up-tables contained entirely in CPU cache. Retrieval of the look-up-table data must never leave the CPU once initially... (6 Replies)
So in my shell i execute:
{ while true; do echo string; sleep 1; done } | read line This waits one second and returns.
But
{ while true; do /bin/echo string; sleep 1; done } | read line continues to run, and doesn't stop until i kill it explicitly.
I have tried this in bash as well as zsh,... (2 Replies)
Hi,
I've modified the syslogd source to include a thread that will keep track of a timer(or a timer thread). My intention is to check the file size of /var/log/messages in every one minute & if the size is more than 128KB, do a echo " " > /var/log/messages, so that the file size will be set... (7 Replies)