Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Echo "${array[@]}" with specified delimiter
Top Forums Shell Programming and Scripting Echo "${array[@]}" with specified delimiter Post 302784219 by Michael Stora on Thursday 21st of March 2013 07:17:46 PM
Old 03-21-2013
Echo "${array[@]}" with specified delimiter
I'm reading in some >41,000 line files and doing some manipulations of columns based on the values of other columns. Arrays make a ton of sense for this application.

Not to slow it down too much, I want to spit out the lines efficiently and not have another loop indexing the entries for each line.

For each row
Code:
$echo "${array[@]}

gives space delimiters (and some of the values contain space so piping it through tr won't work)
Code:
IFS=','; echo "${array[@]}

does not change anything, the output is still space delimited (I was not expecting this to work).

Is there time and processor power efficient way to do this without a loop indexing through all the columns?

Mike

PS. topic related to the code
Code:
IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]} >> out_file
done < in_file

is much much slower than
Code:
(IFS=','; While read -r -a array
Do a bunch of stuff
echo "${array[@]}
done ) < in_file > out_file

I suspect this is because the file is only opened and closed once, not 41,000 times. What are the risks of the subshell over-running it's available memory? I'm looking at ~10 mb files for just Q1 2013 data and I still need to port over data from 2010 on. This will have to be deployed to a Cygwin enviroment.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

"sed" to check file size & echo " " to destination file

Hi, I've modified the syslogd source to include a thread that will keep track of a timer(or a timer thread). My intention is to check the file size of /var/log/messages in every one minute & if the size is more than 128KB, do a echo " " > /var/log/messages, so that the file size will be set... (7 Replies)
Discussion started by: jockey007
7 Replies

2. Shell Programming and Scripting

Difference between using "echo" builtin and /bin/echo

So in my shell i execute: { while true; do echo string; sleep 1; done } | read line This waits one second and returns. But { while true; do /bin/echo string; sleep 1; done } | read line continues to run, and doesn't stop until i kill it explicitly. I have tried this in bash as well as zsh,... (2 Replies)
Discussion started by: ulidtko
2 Replies

3. Programming

C: Initialize "const" array from the "heap"

Hello, I am working on solving an NP-Complete problem, so it is very important that operations and data with limited integer-argument ranges be computed using immutable look-up-tables contained entirely in CPU cache. Retrieval of the look-up-table data must never leave the CPU once initially... (6 Replies)
Discussion started by: HeavyJ
6 Replies

4. Shell Programming and Scripting

With that logic this echoes "echo". Question about echo!

echo `echo ` doesn't echoes anything. And it's logic. But echo `echo `echo ` ` does echoes "echo". What's the logic of it? the `echo `echo ` inside of the whole (first) echo, echoes nothing, so the first echo have to echo nothing but echoes "echo" (too much echoing :P):o (2 Replies)
Discussion started by: hakermania
2 Replies

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

6. AIX

echo $varibla | mail -s "subject" "xxx@xxx.com" not ruuning as expected

Hi Folks, As per the subject, the following command is not working as expected. echo $variable | mail -s "subject" "xxx@xxx.com" Could anyone figure it out whats wrong with this. I am using AIX box. Regards, (2 Replies)
Discussion started by: gjarms
2 Replies

7. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

8. Shell Programming and Scripting

tcsh - understanding difference between "echo string" and "echo string > /dev/stdout"

I came across and unexpected behavior with redirections in tcsh. I know, csh is not best for redirections, but I'd like to understand what is happening here. I have following script (called out_to_streams.csh): #!/bin/tcsh -f echo Redirected to STDOUT > /dev/stdout echo Redirected to... (2 Replies)
Discussion started by: marcink
2 Replies

9. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

LEARN ABOUT NETBSD

cut

CUT(1)							    BSD General Commands Manual 						    CUT(1)

NAME

     cut -- select portions of each line of a file

SYNOPSIS

     cut -b list [-n] [file ...]
     cut -c list [file ...]
     cut -f list [-d delim] [-s] [file ...]

DESCRIPTION

     The cut utility selects portions of each line (as specified by list) from each file and writes them to the standard output.  If the file
     argument is a single dash ('-') or no file arguments were specified, lines are read from the standard input.  The items specified by list can
     be in terms of column position or in terms of fields delimited by a special character.  Column numbering starts from 1.

     list is a comma or whitespace separated set of increasing numbers and/or number ranges.  Number ranges consist of a number, a dash (-), and a
     second number and select the fields or columns from the first number to the second, inclusive.  Numbers or number ranges may be preceded by a
     dash, which selects all fields or columns from 1 to the first number.  Numbers or number ranges may be followed by a dash, which selects all
     fields or columns from the last number to the end of the line.  Numbers and number ranges may be repeated, overlapping, and in any order.	It
     is not an error to select fields or columns not present in the input line.

     The options are as follows:

     -b list	 The list specifies byte positions.

     -c list	 The list specifies character positions.

     -d string	 Use the first character of string as the field delimiter character.  The default is the <TAB> character.

     -f list	 The list specifies fields, separated by the field delimiter character.  The selected fields are output, separated by the field
		 delimiter character.

     -n 	 Do not split multi-byte characters.

     -s 	 Suppresses lines with no field delimiter characters.  Unless specified, lines with no delimiters are passed through unmodified.

EXIT STATUS

     cut exits 0 on success, 1 if an error occurred.

SEE ALSO

     paste(1)

STANDARDS

     The cut utility conforms to IEEE Std 1003.2-1992 (``POSIX.2'').

BSD
								 December 21, 2008							       BSD
All times are GMT -4. The time now is 04:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy