Reading from file bash command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Reading from file bash command
# 1  
Old 09-08-2017
Reading from file bash command

Hello, I have a file in the following format

Code:
id sample platform R1 R2 gene1 gene2 gene3
1       abc     llumina       R1_001.fastq.gz      R2_001.fastq.gz   apoe    prnpp    asp
2       def     llumina       R1_001.fastq.gz      R2_001.fastq.gz   apoe    prnpp    
3       ghi     llumina       R1_001.fastq.gz      R2_001.fastq.gz   apoe

The first 6 columns are always filled, the last two columns are only for some entries. So I have a main script that reads this file to process these ids further. I generally use

Code:
while id sample platform R1 R2 gene1
do
<mainscript.sh>
done < samples.txt

But how can I include the last two columns as well to run the command as many entries under the variable gene2 and gene3 are empty.

This is what i have tried so far to read all the lines in the file and if variable under gene2 and/or gene3 are empty, to move to the next line

Code:
while read  id sample platform R1 R2 gene1 gene2 gene3
do
echo "$id $sample $gene1 $gene2" | {
        read  id sample platform R1 R2 gene1 gene2
        [ -z "$gene2" ] && continue
        for i in $gene2; do eval echo "\"$i\""; done
 }
 
 done < samples.txt

I could do it for gene2 but wasnt able to implement it for gene3

any suggestions would be helpful

thank you
# 2  
Old 09-08-2017
This statement means that if there is no gene2 then there cannot be a gene3 read:
Code:
 [ -z "$gene2" ] && continue

When you use the read statement you have to have all fields there in order to get gene3.

Otherwise gene3 is an "empty" variable. Or the same can happen for gene2.

You can create dummy values for those variables if they are empty, but since you have a separate script processing things that could affect the output of that script.

Try creating an intermediate file with dummy variables, let's use the word 'dummy' for the values of empties. This is awk, the output file will be called tmp.tmp which is what your script will have to use for input.

This could also be set up writing to a pipe which your existing script reads.
Code:
awk '{
        # check to see what type of line we have, 
        # if  col 1 is a number then we have to play the dummy game
        if( int($1) !=0)
        {
            if(NF==6) { $7="dummy"; $8="dummy" }
            if(NF==7) { $8="dummy" }
         }
        print $0 
       }'  inputfile > tmp.tmp

So the last 2 fields in tmp.tmp can be a real value, or just fiber filler: "dummy", but there will always be 8 fields. The last two fields cannot be blank or bash will not read them into a variable.
# 3  
Old 09-08-2017
If you want to add a default/dummy within a bash script, you could do this:-
Code:
my_var="${my_var:-default}"

This says "assign the value of $my_var to itself, or the string default if it is empty or not set." You can have another variable instead of the string default if you want to.


Does that help?



Robin
# 4  
Old 09-09-2017
I do not see the problem.?
Read enough fields, because joining is easier than splitting.
An exercise for demonstration
Code:
while read  id sample platform R1 R2 gene1 gene2 gene3 junk
do
  echo "----"
  echo "process non-empty genes"
  for i in $gene1 $gene2 $gene3; do echo "$i"; done
  echo "process all 3 genes"
  for i in "$gene1" "$gene2" "$gene3"; do echo "$i"; done
done < samples.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash Variable scope - while loop while reading from a file

Cope sample1: test.sh i=0 echo " Outside loop i = $i " while do i=$(( $i + 1)) echo "Inside loop i = $i " done echo " Out of loop i is : $i " When run output : Outside loop i = 0 Inside loop i = 1 Inside loop i = 2 Inside loop i = 3 Inside loop i = 4 Inside loop i = 5 Inside... (8 Replies)
Discussion started by: Adarshreddy01
8 Replies

2. UNIX for Beginners Questions & Answers

Reading a file from a different directory in a Bash script

Hi all, Given here under a section of a script I am using. SIMDIR="/home/Ins/forces" cd $SIMDIR for file in `ls *.forces` do basename=`echo $file | sed 's/\.*$//'` extname=`echo $file | sed 's/*\(*\)\.\(.*\)/\2\1/'` echo "Processing file: "$basename python convert.py... (4 Replies)
Discussion started by: Theo Score
4 Replies

3. Shell Programming and Scripting

Reading a text file using bash

I've a file in linux with following text: ;ip address hostname put-location alt-put-location tftpserver 192.168.1.1 r01-lab1-net /mnt/nas1/fgbu/ /opt/fgbu/devicebackup 192.168.1.254Now I want to read these values and assign them to particular variables... (6 Replies)
Discussion started by: kashif.live
6 Replies

4. Shell Programming and Scripting

Reading command line options from bash script

I have the following code and I am calling it using ./raytrac.bash -u and getting problems. For some reason opt_usage is still 0. opt_usage=0 iarg=0 narg=$# while (($iarg < $narg)) do (( iarg = $iarg + 1 )) arg=$argv usrInputFlag=`echo $arg | awk '/=/ {print 1}; ! /=/... (22 Replies)
Discussion started by: kristinu
22 Replies

5. Shell Programming and Scripting

Bash: Reading a file and assigning variables from file

I have a file that has four values on each line and I'd like to give each column a variable name and then use those values in each step of a loop. In bash, I believe you could use a while loop to do this or possibly a cat command, but I am super new to programming and I'm having trouble decoding... (2 Replies)
Discussion started by: ccorder22
2 Replies

6. Shell Programming and Scripting

Help in reading a cv file in bash

Hi All, I am trying to read a .csv file which has some 6 columns. Eg: samp.csv one, two, three, four six, seven, eight, nine I used the following code, for line in `cat samp.csv` do echo "$line" done It displays every comma seperated values in each line like, one,... (1 Reply)
Discussion started by: johnwilliams.sp
1 Replies

7. Shell Programming and Scripting

Problem in reading file (bash)

i get a name from user first name : last name, in this format. Now i am saving this to a file. what i want is, I do not want to save any name if I already have one entry o that same name..what should i do for example user give robert fernandez this will save in file as robert:fernandez. if... (5 Replies)
Discussion started by: Learnerabc
5 Replies

8. Shell Programming and Scripting

Reading lines from a file, using bash, "at" command

Hi. I have the script shown below. If I execute it form the command line it seems to work properly, but when I fun it using the unix "at" command "at -m now < ./kill-at-job.sh" It appears to hang. Below is the script, the input file, and the execution as reported in the e-mail from the "at"... (3 Replies)
Discussion started by: jbsimon000
3 Replies

9. Shell Programming and Scripting

bash: reading filenames from file

Hi, I'm trying to write a script that reads filenames from a file and use these filenames in a loop. The filenames are all on one line and the problem is that these filenames have wildcards like * and braces like in them. Right now what I'm doing is something like this: echo "reading from... (0 Replies)
Discussion started by: warp17
0 Replies

10. Shell Programming and Scripting

Bash: Reading 2 arguments from a command line

If no arguments are entered I wanna be able to read 2 arguments, i have done like this but it doesnt work: x=0 until #loop starts do if ; then echo No arguments were entered, please enter 2 arguments. read $1 $2 elif || ; then echo $#... (0 Replies)
Discussion started by: Vozx
0 Replies
Login or Register to Ask a Question