Problems with awk (fatal error) and paste (two variables into one column-by-column)
Hello, I have a script extracting columns of useful numbers from a data file, and manipulating the numbers with awk commands. I have problems with my script...
1. There are two lines assigning numbers to $BaseForAveraging. If I use the commented line (the first one) and let the second one commented, it will affect a later awk command because I see the following error:
Code:
awk: cmd. line:1: fatal: cannot open file `{averaged=$3 / BaseForAveraging
printf "%.10f \t %.14f \n",$2,averaged}' for reading (No such file or directory)
However, if I use the second line instead, leaving the first one commented, I will not see the above error.
For either of the above two lines, I see the same number - 1000 when I echo $BaseForAveraging.
My question is... what's wrong with my first line?
2. In a line assigning value to $Data, I have the following error:
Code:
extract_direct_signal.sh: command substitution: line 23: syntax error near unexpected token `('
extract_direct_signal.sh: command substitution: line 23: `paste <(echo "$Data") <(echo "$Temp"))'
My question is ... what's the problem with the command?
I have included a data with which the script works with, so you might try to run it if it is useful to do so...
I have many commented lines at the end because I want to keep them away before getting the above bugs cleared...
I hope you could help me. Thank you very much for your time...
Raymond
===My code===
Code:
#!/bin/bash
#PWLFiles=($(ls $1/*.pwl)) #list the PWL file
PWLFile=SampleDataFile.pwl.txt
#for PWLFile in $PWLFiles
#do
LineNumbers=$(grep -hn "Direct signal" $PWLFile | cut -d ":" -f1) #Extract the Line numbers in which the word "Direct signal" is contained
Data=
for LineNumber in $LineNumbers
do
echo $LineNumber
# BaseForAveraging=$(sed -n "6p" $PWLFile | sed 's/Number of signal records: //g')
BaseForAveraging=1000
Temp=$(sed -n "$((LineNumber + 9)),+999p" $PWLFile | sed 's/ (//g' | awk -v BaseForAveraging=${BaseForAveraging} '{averaged=$3 / BaseForAveraging
printf "%.10f \t %.14f \n",$2,averaged}')
Data=$(paste <(echo "$Data") <(echo "$Temp"))
echo "$Data"
done
# echo "$Data" | awk '{for (j=2;j<=NF;j=j+6)
# {
# for (i=j;i<j+6;i=i+2) total[i]=total[i]+$i
# }
# printf "%.10f \t %.14f \n", $1, total}{total=0}
# for (num in total) printf "%.14f \t", num
# delete total
# printf "\n"' #> $PWLFile.signal
# echo $PWLFile
#done
You need to quote every variable expansion. Likely your $BaseForAveraging ends up with whitespace in it and awk sees that as the program to execute and the next parameter (your actual awk code) as a file to open to reading.
There are a lot of bad practices going on here. Please give us sample input and sample output and you can likely do this a lot cleaner with just 1 invocation of awk.
You need to quote every variable expansion. Likely your $BaseForAveraging ends up with whitespace in it and awk sees that as the program to execute and the next parameter (your actual awk code) as a file to open to reading.
There are a lot of bad practices going on here. Please give us sample input and sample output and you can likely do this a lot cleaner with just 1 invocation of awk.
Hi Neutronscott,
Thanks for your reply.
I have a data file, in which there are 48 repetitions of the following block of data.
In each block, there are 2 parts: "Direct signal" and "Cross-talk" (I have bolded them).
I then extract two sets of information (I underlined them) under only the "Direct signal" section.
1. Number of signal records
2. Data (1st column: time // 2nd column: data value)
Data consists of two columns. I divide each value in the second column by the number of signal records.
Therefore, for each block, I will get 2 columns of data. It is known that the 1st column (time) of each block is the same.
The last step is to add the 2nd columns of every three consecutive blocks up, and tabulate the result. I will get 48/3 = 16 "added" columns. I will finally insert a first column back.
This is what I am trying to implement.
By quoting the variables, do I have to replace every $var with "$var"?
Input:
Code:
% Created 16/09/14 At 16.04.46 < none > SIGNAL "Direct signal, group 1 "
Group 1 consists of:
Wire 1 with label W at (x,y)=(0,0) and at 1600 V
Number of signal records: 1000
Units used: time in second, current in Ampere.
.STIMULUS signal PWL
+ TIME_SCALE_FACTOR = 0.100E-11
+ VALUE_SCALE_FACTOR = 0.100E-11
+ ( 0.00000000E+00 0.00000000E+00
+ 0.30000002E-09 0.00000000E+00### 996 more rows of data here
+ 0.29940000E-06 0.00000000E+00
+ 0.29970002E-06 0.00000000E+00 )
% Created 16/09/14 At 16.04.46 < none > SIGNAL "Cross-talk, group 1 "
Group 1 consists of:
Wire 1 with label W at (x,y)=(0,0) and at 1600 V
Number of signal records: 1000
Units used: time in second, current in Ampere.
.STIMULUS signal PWL
+ TIME_SCALE_FACTOR = 0.100E-11
+ VALUE_SCALE_FACTOR = 0.100E-11
+ ( 0.00000000E+00 0.00000000E+00
+ 0.30000002E-09 0.00000000E+00
### 996 more rows of data here
+ 0.29940000E-06 0.00000000E+00
+ 0.29970002E-06 0.00000000E+00 )
Output:
Code:
0.00000000E+00 0.00000000000E+00 ###14 more columns here 0.00000000000E+00
0.30000002E+00 0.00000000001E+00 ###14 more columns here 0.00000000001E+00
### 996 more rows of data here
0.29940000E+00 0.00000000005E+00 ###14 more columns here 0.00000000005E+00
0.29970002E+00 0.00000000002E+00 ###14 more columns here 0.00000000002E+00
You could try this to get at the (two in your sample file) data columns (unfortunately all zeroes in your sample file); I didn't quite understand what averaging you wanted to achieve, so I leave it up to you. Give it a try:
You could try this to get at the (two in your sample file) data columns (unfortunately all zeroes in your sample file); I didn't quite understand what averaging you wanted to achieve, so I leave it up to you. Give it a try:
Thank you so much for offering the following code. That is so neat and useful for me. I have understood it with some online tutorial. awk is great.
Raymond
Quote:
Originally Posted by RudiC
You could try this to get at the (two in your sample file) data columns (unfortunately all zeroes in your sample file); I didn't quite understand what averaging you wanted to achieve, so I leave it up to you. Give it a try:
I have data of an excel files as given below,
file1
org1_1 1 1 2.5 100
org1_2 1 2 5.5 98
org1_3 1 3 7.2 88
file2
org2_1 1 1 2.5 100
org2_2 1 2 5.5 56
org2_3 1 3 7.2 70
I have multiple excel files as above shown.
I have to copy column 1, column 4 and paste into a new excel file as... (26 Replies)
Example:
I have files in below format
file 1:
zxc,133,joe@example.com
cst,222,xyz@example1.com
File 2 Contains:
hxd
hcd
jws
zxc
cst
File 1 has 50000 lines and file 2 has around 30000 lines :
Expected Output has to be :
hxd
hcd
jws (5 Replies)
Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2
Scenario 1
i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of... (1 Reply)
Hi Experts,
Please bear with me, i need help
I am learning AWk and stuck up in one issue.
First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique.
Second point : For... (1 Reply)
Any shortcuts for doing this? I need to cut the column 4 values from File1 and paste them as column4 values of File2, but only for the (first) same number of lines as File1 . All rows in File1 are contained in File2 in the exact same order, so the cut paste should work.
File1 (with header and 3... (4 Replies)
Hi,
My input files is like this
axis1 0 1 10
axis2 0 1 5
axis1 1 2 -4
axis2 2 3 -3
axis1 3 4 5
axis2 3 4 -1
axis1 4 5 -6
axis2 4 5 1
Now, these are my following tasks
1. Print a first column for every two rows that has the same value followed by a string.
2. Match on the... (3 Replies)
Hey folks. I wrote a little awk script that summarizes /proc/net/dev info and then pipes it to the nix column command to set up column spacing appropriately.
Here's some example output:
Iface RxMBytes RxPackets RxErrs RxDrop TxMBytes TxPackets TxErrs TxDrop
bond0 9 83830... (3 Replies)
Hi,
I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column.
clientname,day-of-month,max-users
-----------------------------------
client1,20120610,5
client2,20120610,2
client3,20120610,7... (3 Replies)
Hello,
I have 2 columns (1st column has multiple entries but the corresponding values in the column 2 may be the same or different.) however I want to extract unique values for each entry in column 1 by assigning the max value from column 2
SDF4 -0.211654
SDF4 0.978068
... (1 Reply)
Hi,
I need to paste each 10 lines of single column to several columns.
Please, can anyone tell me how to write in awk?
Input File:
22
34
36
12
17
19
15
11
89
99
56
38
29 (4 Replies)
10-27-2014
