Shell Programming and Scripting

I use a perl script to add specific text to specific columns in a text file. A new file gets created in the location listed below but it is empty. So I am not sure if the script itself is incorrect or if the input file is not being used. Below is how I call the perl script:

I provide the full path to the .pl , the input and the output

Code:

 perl 'C:\Users\cmccabe\Desktop\annovar\matrix.pl' < "${id}".txt.hg19_multianno.txt > "L:\NGS\3_BUSINESS\Matrix\Torrent\matrix_"${id}".txt"

The overall goal is use ${id}.txt.hg19_multianno.txt as the input file and after the perl script is run on that file (which adds the columns and text) and the new file is saved to the path L:\NGS\3_BUSINESS\Matrix\Torrent\matrix_${id}.txt . That path is a static path and will never change, there is probably a better way but I am not expert enough to know. I attached the .pl as well. Thank you

Your Perl program is broken. The first 49 "if" statements do not have their closing braces. A part of your script is shown below with the problem:

Code:

  ...
  ...
  if ( $. == 1 ) {  <== this opening brace does not have a corresponding closing brace
    s/$/ Index/;
  if ( $. == 2 ) { <== this opening brace does not have a corresponding closing brace
    s/$/ Chromosome Position/;
  ...
  ...

As a result the Perl script, when run, will fail and abort with a compile time error similar to the following (I've put the 3 dots below (...), but you'll see some information instead of the dots):

Code:

Missing right curly or square bracket at ...
syntax error at ...
Execution of ... aborted due to compilation errors.

(1) The reason you are seeing a file in your L:\NGS location is because that is done by your shell. When you put a redirection operator "> some_file", the shell immediately creates a 0-byte file at that moment in that location.

(2) And the reason the file at L:\NGS is empty is because the Perl program does not work. It fails with the error and does not print anything in the standard output (STDOUT) that could go to the L:\NGS file.

Now, you should be able to see the error message thrown by Perl. I'm not sure why you're not.

Maybe you are calling your Bash shell script like so?

Code:

my_shell_script.sh 2>/dev/null

That will redirect all error messages into the NULL device and you won't see the error message spewed by Perl.

To fix the problems:

(1) Ensure that the Perl program is correct syntax-wise. All opening braces must have corresponding closing braces.

(2) Replace this line in your shell script:

Code:

perl 'C:\Users\cmccabe\Desktop\annovar\matrix.pl' < "${id}".txt.hg19_multianno.txt > "L:\NGS\3_BUSINESS\Matrix\Torrent\matrix_"${id}".txt"

by this:

Code:

perl 'C:\Users\cmccabe\Desktop\annovar\matrix.pl' < "${id}".txt.hg19_multianno.txt

and run your shell script without the "2 >/dev/null" (if it is so now).
That should print the output to your screen (terminal). If it looks satisfactory, then add the "> L:\NGS\..." part to it.

Thank you, the perl scripts runs but the results are not as expected. The .pl attached is supposed to add the text to the specific columns, but it is not and I'm not sure why or if there is a better way.

Basically, ${id}.txt.hg19_multianno.txt is the input file and after the perl script is run on that file (which adds the columns and text) and the new file is saved to the path L:\NGS\3_BUSINESS\Matrix\Torrent\matrix_${id}.txt

I also attached the desired results as an excel with the yellow color being the text/fields added by the script and the green are the text/fields in the original file. Thank you .

In the Perl script you add strings to the end of specific lines.

$. is the line number

$ in a substitution is the end of the current line.

So you're adding strings to the end of some lines rather than amending columns.

Refering to your "output.xls" file in the previous post, are the following assumptions correct?

(1) The data in the range S1:AP87 in the "Test" worksheet is in your "${id}.txt.hg19_multianno.txt" file already.

(2) You want to add the data in the range A1:R87 to the left of the existing data.

(3) You want to add the data in the range AQ1:AX87 to the right of the existing data.

If the assumptions above are correct, then:

(a) From where do you get the data in the range A2:R87?
(b) And from where do you get the data in the range AQ2:AX87?

Only the header text is needed. The attempted perl script was only going to add these columns.

So basically,

Code:

 A1 = Index, B1 = Chromosomal Position, C1 = etc...

The only columns with data come from "${id}.txt.hg19_multianno.txt". Thank you .

I don't think I understood this statement:
Do you mean:

(a) the "${id}.txt.hg19_multianno.txt" file has only one line similar to this?

Code:

Chr Start ... clinvarsubmit clinvarreference

And you want to change it to this?

Code:

Index Chromosome ... Amino Acid Change Chr Start ... clinvarsubmit clinvarreference HP SPLICE ... Sanger References

Or

(b) the "${id}.txt.hg19_multianno.txt" file has a header line like this?

Code:

Chr Start ... clinvarsubmit clinvarreference

and data (from line no. 2 onwards) similar to the range A2:AX87 in the "Test" worksheet of your Excel workbook "output.xls".
And you want to convert the header to this:

Code:

Index Chromosome ... Amino Acid Change Chr Start ... clinvarsubmit clinvarreference HP SPLICE ... Sanger References

and keep the data as it is (i.e. just iterate from line no. 2 onwards till end of file without making any changes)?