Sponsored Content
Top Forums Shell Programming and Scripting Echo "${array[@]}" with specified delimiter Post 302784219 by Michael Stora on Thursday 21st of March 2013 07:17:46 PM
Old 03-21-2013
Echo "${array[@]}" with specified delimiter

I'm reading in some >41,000 line files and doing some manipulations of columns based on the values of other columns. Arrays make a ton of sense for this application.

Not to slow it down too much, I want to spit out the lines efficiently and not have another loop indexing the entries for each line.

For each row
Code:
$echo "${array[@]}

gives space delimiters (and some of the values contain space so piping it through tr won't work)
Code:
IFS=','; echo "${array[@]}

does not change anything, the output is still space delimited (I was not expecting this to work).

Is there time and processor power efficient way to do this without a loop indexing through all the columns?

Mike

PS. topic related to the code
Code:
IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]} >> out_file
done < in_file

is much much slower than
Code:
(IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]}
done ) < in_file > out_file

I suspect this is because the file is only opened and closed once, not 41,000 times. What are the risks of the subshell over-running it's available memory? I'm looking at ~10 mb files for just Q1 2013 data and I still need to port over data from 2010 on. This will have to be deployed to a Cygwin enviroment.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

"sed" to check file size & echo " " to destination file

Hi, I've modified the syslogd source to include a thread that will keep track of a timer(or a timer thread). My intention is to check the file size of /var/log/messages in every one minute & if the size is more than 128KB, do a echo " " > /var/log/messages, so that the file size will be set... (7 Replies)
Discussion started by: jockey007
7 Replies

2. Shell Programming and Scripting

Difference between using "echo" builtin and /bin/echo

So in my shell i execute: { while true; do echo string; sleep 1; done } | read line This waits one second and returns. But { while true; do /bin/echo string; sleep 1; done } | read line continues to run, and doesn't stop until i kill it explicitly. I have tried this in bash as well as zsh,... (2 Replies)
Discussion started by: ulidtko
2 Replies

3. Programming

C: Initialize "const" array from the "heap"

Hello, I am working on solving an NP-Complete problem, so it is very important that operations and data with limited integer-argument ranges be computed using immutable look-up-tables contained entirely in CPU cache. Retrieval of the look-up-table data must never leave the CPU once initially... (6 Replies)
Discussion started by: HeavyJ
6 Replies

4. Shell Programming and Scripting

With that logic this echoes "echo". Question about echo!

echo `echo ` doesn't echoes anything. And it's logic. But echo `echo `echo ` ` does echoes "echo". What's the logic of it? the `echo `echo ` inside of the whole (first) echo, echoes nothing, so the first echo have to echo nothing but echoes "echo" (too much echoing :P):o (2 Replies)
Discussion started by: hakermania
2 Replies

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

6. AIX

echo $varibla | mail -s "subject" "xxx@xxx.com" not ruuning as expected

Hi Folks, As per the subject, the following command is not working as expected. echo $variable | mail -s "subject" "xxx@xxx.com" Could anyone figure it out whats wrong with this. I am using AIX box. Regards, (2 Replies)
Discussion started by: gjarms
2 Replies

7. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

8. Shell Programming and Scripting

tcsh - understanding difference between "echo string" and "echo string > /dev/stdout"

I came across and unexpected behavior with redirections in tcsh. I know, csh is not best for redirections, but I'd like to understand what is happening here. I have following script (called out_to_streams.csh): #!/bin/tcsh -f echo Redirected to STDOUT > /dev/stdout echo Redirected to... (2 Replies)
Discussion started by: marcink
2 Replies

9. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies
CUT(1)							      General Commands Manual							    CUT(1)

NAME
cut - select out columns of a file SYNOPSIS
cut [ -b | -c] list [file...] cut -f list [-d delim] [ -s] OPTIONS
-b Cut specified bytes -c Select out specific characters -d Change the column delimiter to delim -f Select out specific fields that are separated by the -i Runs of delimiters count as one -s Suppres lines with no delimiter characters, when used EXAMPLES
cut -f 2 file # Extract field 2 cut -c 1-2,5 file # Extract character columns 1, 2, and 5 cut -c 1-5,7- file # Extract all columns except 6 DESCRIPTION
[file...]" delimiter character ( see delim)" with the -f option. Lines with no delimiters are passwd through untouched" Cut extracts one or more fields or columns from a file and writes them on standard output. If the -f flag is used, the fields are sepa- rated by a delimiter character, normally a tab, but can be changed using the -d flag. If the -c flag is used, specific columns can be specified. The list can be comma or BLANK separated. The -f and -c flags are mutually exclusive. Note: The POSIX1003.2 standard requires the option -b to cut out specific bytes in a file. It is intended for systems with multi byte characters (e.g. kanji), since MINIX uses only one byte characters, this option is equivalent to -c. For the same reason, the option -n has no effect and is not listed in this man- ual page. SEE ALSO
sed(1), awk(9). CUT(1)
All times are GMT -4. The time now is 05:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy