Sponsored Content
Top Forums Shell Programming and Scripting Read files, lines into array, cat vs open Post 302370606 by jimmy_y on Thursday 12th of November 2009 02:14:39 AM
Old 11-12-2009
Read files, lines into array, cat vs open

Hi Everyone,

I have a file: a.txt
a,b,c,d,6,6,6
1,2,3,d,6,6,6
4,5,6,6,6,6,6

Code:
#!/usr/bin/perl
use warnings;
use strict;
my @array = ();
### Load file into array
for my $i (split '\n', `cat /tmp/a.txt`) {
    push @array, [ split ',', $i ];
}

It works. But my a.txt have 1million lines, and each lines have 17 fields. If use 'cat' like above, it takes veeeeeeeeeery long time.

How to do it using open?, especially need to split ',' for each line.Smilie

Thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help!! Need script to read files and add values by lines...

Hi All, I really need your help. I am a begginner in shell script and I believe this is a very simple issue. I have in my directory, n-files, like 1.dhm, 2.dhm, 3.dhm. These files have 1 column with 1 value per line, like: 1.dhm ------ 10 20 30 40 50 2.dhm ------ 30 50 20 (3 Replies)
Discussion started by: dhuertas
3 Replies

2. Shell Programming and Scripting

cat file1 read line-per-line then grep -A 15 lines down in fileb

STEP 1 # Set variable FILE=/tmp/mainfile SEARCHFILE =/tmp/searchfile # THIS IS THE MAIN FILE. cat /tmp/mainfile Interface Ethernet0/0 "outside", is up, line protocol is up Hardware is i82546GB rev03, BW 100 Mbps Full-Duplex(Full-duplex), 100 Mbps(100 Mbps) MAC address... (6 Replies)
Discussion started by: irongeekio
6 Replies

3. Shell Programming and Scripting

help using read in menu script to cat out lines in logs

What is wrong with my menu script? Do I need to continue with the read statements? All I want to do with option 4 is to cat some /var/log/files and awk out a few lines? How do I do that please? $ cat menu.sh ... (11 Replies)
Discussion started by: taekwondo
11 Replies

4. Shell Programming and Scripting

How to read multiple lines from Std Input into an array

Hi All, Does anyone know how to read multiple lines from standard input into an array and then iterate a loop for all the lines read. Below is an example pseudocode: I need the below filenames to be read by the script into an array or something similar: And then in the script, I... (9 Replies)
Discussion started by: bharath.gct
9 Replies

5. UNIX for Dummies Questions & Answers

Read directory files and count number of lines

Hello, I'm trying to create a BASH file that can read all the files in my working directory and tell me how many words and lines are in that file. I wrote the following code: FILES="*" for f in "$FILES" do echo -e `wc -l -w $f` done My issue is that my file is outputting in one... (4 Replies)
Discussion started by: jl487
4 Replies

6. UNIX for Advanced & Expert Users

cat can not open file

Hi All, I have stumbled upon very unique issue. In my script I am doing cat file and then greping and cutting so as to assign the value to variable. My file is, <mxc_tl_load_extractdata_prop.bsh> DB_USER=test_oper hostname=xxx FTP_USER=test1_operate MAIL_LIST=xxx@yyy.com... (1 Reply)
Discussion started by: paragd
1 Replies

7. Shell Programming and Scripting

PERL : Read an array and write to another array with intial string pattern checks

I have an array and two variables as below, I need to check if $datevar is present in $filename. If so, i need to replace $filename with the values in the array. I need the output inside an ARRAY How can this be done. Any help will be appreciated. Thanks in advance. (2 Replies)
Discussion started by: irudayaraj
2 Replies

8. Shell Programming and Scripting

How can I read variable text files through cat command?

Hi. I'm just wondering how can I read variable text files through cat command. I made a shell script to count frequency of words and array them with variable conditions. It's only working for one file that I wrote in script now. But I want to make it working for every text file when I execute... (2 Replies)
Discussion started by: rlaxodus
2 Replies

9. Shell Programming and Scripting

read the lines of multiple files

I am trying to create a script which will read 2 files and use the lines of file 1 for each line on file 2. here's my sample code cat $SBox | while read line do cat $Date | while read line do $SCRIPTEXE <line from first file> $2 <line from 2nd file> ... (12 Replies)
Discussion started by: khestoi
12 Replies

10. Shell Programming and Scripting

How do I cat into an array or is it not possible?

Hi, Normally, I would do cat /etc/oratab | grep -v "^#" | grep -v "^*" > /tmp/oratab.00 while read line do echo $line done < /tmp/oratab.00I want to know whether it is possible to use an array instead of re-directing it to file? As a test, I did something like below: #!/bin/ksh ... (6 Replies)
Discussion started by: newbie_01
6 Replies
split(1)						      General Commands Manual							  split(1)

NAME
split - split a file into pieces SYNOPSIS
line_count] suffix_length] [file [name]] nsuffix_length] [file [name]] Obsolescent n] [file [name]] DESCRIPTION
reads file and writes it in pieces (default 1000 lines) onto a set of output files. The name of the first output file is name with appended, and so on lexicographically, up to (only ASCII letters are used, a maximum of 676 files). If no output name is given, is the default. If no input file is given, or if is given instead, the standard input file is used. Options recognizes the following command-line options and arguments: The input file is split into pieces line_count lines in size. suffix_length letters are used to form the suffix of the output filenames. This option allows creation of more than 676 output files. The output file names created cannot exceed the maximum file name length allowed in the directory containing the files. The input file is split into pieces n bytes in size. The input file is split into pieces n x 1024 bytes in size. No space separates the n from the The input file is split into pieces n x 1048576 bytes in size. No space separates the n from the The input file is split into pieces n lines in size. This option is obsolescent and is equivalent to using the option. EXTERNAL INFLUENCES
Environment Variables determines the locale for the interpretation of text as single- and/or multi-byte characters. determines the language in which messages are displayed. If or is not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty variable. If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are set to "C". See environ(5). International Code Set Support Single- and multi-byte character code sets are supported. SEE ALSO
csplit(1). STANDARDS CONFORMANCE
split(1)
All times are GMT -4. The time now is 07:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy