Read files, lines into array, cat vs open


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read files, lines into array, cat vs open
# 1  
Old 11-12-2009
Read files, lines into array, cat vs open

Hi Everyone,

I have a file: a.txt
a,b,c,d,6,6,6
1,2,3,d,6,6,6
4,5,6,6,6,6,6

Code:
#!/usr/bin/perl
use warnings;
use strict;
my @array = ();
### Load file into array
for my $i (split '\n', `cat /tmp/a.txt`) {
    push @array, [ split ',', $i ];
}

It works. But my a.txt have 1million lines, and each lines have 17 fields. If use 'cat' like above, it takes veeeeeeeeeery long time.

How to do it using open?, especially need to split ',' for each line.Smilie

Thanks
# 2  
Old 11-12-2009
Code:
open my $FH, '<', '/tmp/a.txt' or die "Can't open a.txt: $!";
while ( my $line = <$FH> ) {
    chomp $line; # remove the newline at the end
    push @array, [ split ',', $line ];
}
close $FH;

Or (perlisher)
Code:
open my $FH, '<', 'a.txt';
@array = map { chomp; [ split ',' ] } <$FH>;
close $FH;

# 3  
Old 11-12-2009
[/COLOR]Smilie much much more faster now! Thanks Smilie

Last edited by jimmy_y; 11-12-2009 at 06:44 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do I cat into an array or is it not possible?

Hi, Normally, I would do cat /etc/oratab | grep -v "^#" | grep -v "^*" > /tmp/oratab.00 while read line do echo $line done < /tmp/oratab.00I want to know whether it is possible to use an array instead of re-directing it to file? As a test, I did something like below: #!/bin/ksh ... (6 Replies)
Discussion started by: newbie_01
6 Replies

2. Shell Programming and Scripting

read the lines of multiple files

I am trying to create a script which will read 2 files and use the lines of file 1 for each line on file 2. here's my sample code cat $SBox | while read line do cat $Date | while read line do $SCRIPTEXE <line from first file> $2 <line from 2nd file> ... (12 Replies)
Discussion started by: khestoi
12 Replies

3. Shell Programming and Scripting

How can I read variable text files through cat command?

Hi. I'm just wondering how can I read variable text files through cat command. I made a shell script to count frequency of words and array them with variable conditions. It's only working for one file that I wrote in script now. But I want to make it working for every text file when I execute... (2 Replies)
Discussion started by: rlaxodus
2 Replies

4. Shell Programming and Scripting

PERL : Read an array and write to another array with intial string pattern checks

I have an array and two variables as below, I need to check if $datevar is present in $filename. If so, i need to replace $filename with the values in the array. I need the output inside an ARRAY How can this be done. Any help will be appreciated. Thanks in advance. (2 Replies)
Discussion started by: irudayaraj
2 Replies

5. UNIX for Advanced & Expert Users

cat can not open file

Hi All, I have stumbled upon very unique issue. In my script I am doing cat file and then greping and cutting so as to assign the value to variable. My file is, <mxc_tl_load_extractdata_prop.bsh> DB_USER=test_oper hostname=xxx FTP_USER=test1_operate MAIL_LIST=xxx@yyy.com... (1 Reply)
Discussion started by: paragd
1 Replies

6. UNIX for Dummies Questions & Answers

Read directory files and count number of lines

Hello, I'm trying to create a BASH file that can read all the files in my working directory and tell me how many words and lines are in that file. I wrote the following code: FILES="*" for f in "$FILES" do echo -e `wc -l -w $f` done My issue is that my file is outputting in one... (4 Replies)
Discussion started by: jl487
4 Replies

7. Shell Programming and Scripting

How to read multiple lines from Std Input into an array

Hi All, Does anyone know how to read multiple lines from standard input into an array and then iterate a loop for all the lines read. Below is an example pseudocode: I need the below filenames to be read by the script into an array or something similar: And then in the script, I... (9 Replies)
Discussion started by: bharath.gct
9 Replies

8. Shell Programming and Scripting

help using read in menu script to cat out lines in logs

What is wrong with my menu script? Do I need to continue with the read statements? All I want to do with option 4 is to cat some /var/log/files and awk out a few lines? How do I do that please? $ cat menu.sh ... (11 Replies)
Discussion started by: taekwondo
11 Replies

9. Shell Programming and Scripting

cat file1 read line-per-line then grep -A 15 lines down in fileb

STEP 1 # Set variable FILE=/tmp/mainfile SEARCHFILE =/tmp/searchfile # THIS IS THE MAIN FILE. cat /tmp/mainfile Interface Ethernet0/0 "outside", is up, line protocol is up Hardware is i82546GB rev03, BW 100 Mbps Full-Duplex(Full-duplex), 100 Mbps(100 Mbps) MAC address... (6 Replies)
Discussion started by: irongeekio
6 Replies

10. Shell Programming and Scripting

Help!! Need script to read files and add values by lines...

Hi All, I really need your help. I am a begginner in shell script and I believe this is a very simple issue. I have in my directory, n-files, like 1.dhm, 2.dhm, 3.dhm. These files have 1 column with 1 value per line, like: 1.dhm ------ 10 20 30 40 50 2.dhm ------ 30 50 20 (3 Replies)
Discussion started by: dhuertas
3 Replies
Login or Register to Ask a Question