Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Read lines with different lengths in while loop

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read lines with different lengths in while loop
# 1  
Old 04-13-2010
Read lines with different lengths in while loop
Hi there !

I need to treat files with variable line length, and process the tab-delimited words of each line. The tools I know are some basic bash scripting and sed ... I haven't got to python or perl yet.

So my file looks like this

Code:
obj1 0.01953 0.34576 0.04418 0.01249
obj2 0.78140 0.00015 0.02175 0.19468 0.03274
obj3 0.00014 0.00016 0.00014 1.03222 0.24256 0.00014 0.00014 0.00014 0.00016 
etc

I'd like to sum all numbers following each "obj".

I've tried something like this, but that (logically) reads over the line breaks :

Code:
sum=0
while read -d $'\t' brlen
do
if [ ${brlen:0:3} = "obj" ]
then
    if [ ! $sum = "0" ]
    then
        echo $sum
    fi
    echo -e -n "${brlen}\t"
    sum=0
else
    temp=`echo "scale=3; ${sum}+${brlen}" | bc -l`
    sum=$temp
fi
done < myfile

Of course, a solution would be to create a different file with

Code:
sed -e "s/\t/\n/g" myfile > myfile1

but can I do this directly ?
Can I feed the output of the sed command directly into the while loop, for instance ?

Thanks in advance !
# 2  
Old 04-13-2010
Not sure if this is what you're looking for, but you can try this:
Code:
awk '{for(i=2;i<=NF;i++){s+=$i}; print $1, s;s=0}' file

Regards
# 3  
Old 04-13-2010
Great !

It works, and it's (not surprisingly) much faster than my bc-based solution.
I didn't know that awk could do floating-point arithmetic ... I'm going to learn how to use it right away !

thanks !
jos
# 4  
Old 04-13-2010
Hai here its a Perl solution.
Code:
use strict;
use warnings;

#opening the file for reading
open FH,"<data_input" or die "Can't open file : $!\n";
my @line;
while ( <FH> )
{
        if ( /(obj[1-9]*)/ )
        {
                my $line = $';
                my $val=0;
                @line = split(' ',$line);
                $val += $_ for @line;
                print "Sum : $val\n";
        }
}

The output will be

Code:
Sum : 0.42196
Sum : 1.03072
Sum : 1.2758

# 5  
Old 04-13-2010
MySQL
You can try this..

Code:
[root@sistem1lnx ~]# cat file
 
obj1 0.01953 0.34576 0.04418 0.01249
obj2 0.78140 0.00015 0.02175 0.19468 0.03274
obj3 0.00014 0.00016 0.00014 1.03222 0.24256 0.00014 0.00014 0.00014 0.00016
 
[root@sistem1lnx ~]# i=1;filecount=`cat file|wc -l`;for i in $(seq 1 $filecount);do 
  sed -n '/obj'$i'/,/obj'$((i+1))'/{;/obj'$((i+1))'/!p;}' file 
| sed -e 's/^[a-zA-Z][a-zA-Z]*[0-9][0-9]*//g' 
| sed -e 's/ /\+ /g' -e 's/^.//g' | bc ; i+=1 ;done 
| sed 's/^\./0./g' 
| sed =  
| sed -e N -e 's/^/     /; s/ *\(.\{1,\}\)\n/\1  /'
 
1  0.42196
2  1.03072
3  1.27580

# 6  
Old 04-13-2010
Quote:
Originally Posted by ygemici
You can try this..

Code:
[root@sistem1lnx ~]# cat file
 
obj1 0.01953 0.34576 0.04418 0.01249
obj2 0.78140 0.00015 0.02175 0.19468 0.03274
obj3 0.00014 0.00016 0.00014 1.03222 0.24256 0.00014 0.00014 0.00014 0.00016
 
[root@sistem1lnx ~]# i=1;filecount=`cat file|wc -l`;for i in $(seq 1 $filecount);do 
  sed -n '/obj'$i'/,/obj'$((i+1))'/{;/obj'$((i+1))'/!p;}' file 
| sed -e 's/^[a-zA-Z][a-zA-Z]*[0-9][0-9]*//g' 
| sed -e 's/ /\+ /g' -e 's/^.//g' | bc ; i+=1 ;done 
| sed 's/^\./0./g' 
| sed =  
| sed -e N -e 's/^/     /; s/ *\(.\{1,\}\)\n/\1  /'
 
1  0.42196
2  1.03072
3  1.27580

Not the simplest, nor the most effective way to do the task with a cat, a wc, a seq and 6 sed commands for every line Smilie
# 7  
Old 04-13-2010
MySQL
Quote:
Originally Posted by Franklin52
Not the simplest, nor the most effective way to do the task with a cat, a wc, a seq and 6 sed commands for every line Smilie
yes you are right but sed way is this Smilie
best solution your awk Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Paste files of varying lengths

I have three files of varying lengths and different number of columns. How can I paste all three with all columns aligned? File1 ---- 123 File2 ---- 234 345 678 File3 ---- 456 789 Output should look like: 123 234 456 345 789 (6 Replies)
Discussion started by: Un1xNewb1e
6 Replies

2. Shell Programming and Scripting

Merge two files with different lengths

Hi there, I have two very long files like: file1: two fields 1 123 1 125 1 234 2 123 2 234 2 300 2 312 3 10 3 215 4 56 ... (11 Replies)
Discussion started by: ClaraW
11 Replies

3. Shell Programming and Scripting

Re: Read lines and compare in a loop

I have a file which has following content: NAME=ora.DG1.dg TYPE=ora.diskgroup.type TARGET=ONLINE STATE=ONLINE NAME=ora.DG2.dg TYPE=ora.diskgroup.type TARGET=ONLINE STATE=ONLINE NAME=ora.DG3.dg TYPE=ora.diskgroup.type TARGET=ONLINE STATE=ONLINE NAME=ora.DG4.dg... (7 Replies)
Discussion started by: rcc50886
7 Replies

4. Shell Programming and Scripting

Reading two lines in a while loop and combining the lines

Dear all, I have a file like this: imput scaffold_0 1 scaffold_0 10000 scaffold_0 20000 scaffold_0 25000 scaffold_1 1 scaffold_1 10000 scaffold_1 20000 scaffold_1 23283 and I want the output like this: scaffold_0 1 scaffold_0 10000 scaffold_0 10000 scaffold_0 20000... (6 Replies)
Discussion started by: valente
6 Replies

5. UNIX for Dummies Questions & Answers

When reading a csv file, counter to read 20 lines and wait for minute then read next 20 till end

Hello All, i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file. Any inputs are appreciated ... (3 Replies)
Discussion started by: victor.s
3 Replies

6. Shell Programming and Scripting

Merging data from 2 files of different lengths?

Hi all, Sorry if someone has answered something like this already, but I have a problem. I am not brilliant with "awk" but think it should be the command to use to get what I am after. I have 2 files: job-file (several hundred lines like): 1018003,LONG MU WAN,1113S 1018004,LONG MU... (4 Replies)
Discussion started by: sgb2301
4 Replies

7. UNIX for Dummies Questions & Answers

Read statement within while read loop

hi, this is my script #!/bin/ksh cat temp_file.dat | while read line do read test if ]; then break else echo "ERROR" fi done when i execute this code , the script does wait for the user input . it directly prints "ERROR" and terminates after the no. of times as there... (3 Replies)
Discussion started by: siva1612
3 Replies

8. Shell Programming and Scripting

Having a for loop read in lines with spaces?

Is this possible? I have a for loop in a shell script reading a list, but I want each line to be a loop, not each thing with a space. Here is the example: HOSTLIST="\ 1.2.3.4 serverA 1.2.3.5 serverB" for NBUHOST in `echo $HOSTLIST` do ssh ${SERVERNAME} "echo "${NBUHOST}"... (3 Replies)
Discussion started by: LordJezoX
3 Replies

9. Shell Programming and Scripting

How to use while loop in bash shell to read a file with 4 lines of gap

Hi , I am currently using the while loop in bash shell, as follows. while read line do echo $line done < file.txt However, i want to use the while loop on file.txt, which will read the file with 4 lines of gap. Ex- if file.txt is a file of 100 lines, then i want to use the loop such... (3 Replies)
Discussion started by: jitendriya.dash
3 Replies

10. UNIX for Dummies Questions & Answers

Using grep to find strings of certain lengths?

I am trying to use grep to find strings of certain lengths that all start with the same letter. Is this possible?:confused: (4 Replies)
Discussion started by: crabtruck
4 Replies
Login or Register to Ask a Question