Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to insert gaussian noise to a data with an order with awk?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to insert gaussian noise to a data with an order with awk?
# 22  
Old 07-24-2012
Quote:
Originally Posted by miriammiriam
Only there are,this time, the spaces between station information and the time information increased and there are only 2 digits for the time information the rest is the same
The rest is not the same.

The first header is no longer the same length as subsequent headers. Lines with less than 6 stations do not have subsequent fields with zero values; they are now absent. The mark line between events which contained a 0 in each field is now a single zero.

Does the file now contain nullbytes? I ask because that would not be a good thing, because ^@ is the usual visual representation of a nullbyte, and because your sample data now includes 24-Jul-12 ^@^@^@^@^@^@^@^@.

If the file format isn't set in stone, I would suggest implementing something that's easier to parse. Use a tab to delimit fields. Have only a single station and time value per line. A blank line can delimit events, instead of a line with a zero (or multiple zeroes). Or perhaps no marker is required. If each station line consisted of two fields (station name <tab> time), and if the header is always guaranteed to be more than 2 fields, the field count would be sufficient.

A simple format, after any necessary processing (such as adding the noise), can be easily converted to another format (perhaps of fixed-width if that's a requirement). Just a thought.

Regards,
Alister

---------- Post updated at 10:02 AM ---------- Previous update was at 09:55 AM ----------

Quote:
Originally Posted by raj_saini20
Try this (tested in hp-ux)

Code:
<insert a long line of inscrutible code here>

Please, in the future, avoid such long lines and format the code with proper indentation (there are at least 3 levels of nesting in there). Even for an expert, they are difficult to comprehend. For a novice, nearly impossible. To make matters worse, unless someone is using a very wide display resolution and has the browser maximized, your post and every post that quotes yours requires tedious scrolling to read.

Regards,
Alister
Last edited by alister; 07-24-2012 at 11:21 AM..
# 23  
Old 07-25-2012
Code:
0910 1 2048 32.38 40N20.01  28W36.24  10.85   0.00                   32.00     computed  24-Jul-12 ^@^@^@^@^@^@^@^@
MSDMP0    3.49KCTXP2    6.60KRCMP0    6.63BOZMP0    6.94ARMTP2    8.03FRR6P0    9.30
CINRP0   10.33FRR7P0   10.20TRNMP0   14.25CEKMP0   17.68ATIMP2   18.06KRBGP3   21.01
0
0910 1 2328 55.48 40N41.07  29W13.91  12.06   0.00                   55.35
KLCMP0    4.92SAKIP0    5.03ARCEP0    5.59GBZMP0    5.88IBBTP0    5.30BUYMP0    5.36
GOZTP0    5.94CINRP0    5.50KRMLP0    6.55SABAP0    6.86ESKMP0    6.42GEMTP2    6.44
MOLLP0    7.38KURNP0    7.68FRONP0    6.61ARMTP2    7.71CEKMP0    8.13FRR7P0    7.62
BOZMP0    8.91ISK P2    9.23IGDMP0    9.41MDNYP2    9.64ADVTP2   10.78FRR6P0   10.92
SILTP2   12.62KLYTP2   12.65MSDMP0   13.14BGKTP2   13.06ELBAP0   16.22CTKSP3   16.26
KCTXP3   17.35KRCMP1   17.37SNLMP1   19.64SLVTP3   19.66TRNMP1   22.69GONEP3   26.98
KRBGP3   29.44RKY P3   29.98LAP P3   35.99
0

Code:
computed  24-Jul-12 ^@^@^@^@^@^@^@^@

This part can be eliminated from the file it is not so much important and the rest of the file does not contain any signs like this.

I know that the header lines are not the same as before but the that given says me,ı guess, process the lines only starting with letters, else write the whole line as it is. So I thought that it is not important that the line length has changed... You would be right i am not so sure Smilie
# 24  
Old 07-25-2012
try this
Code:
$awk 'BEGIN{f1=0;i=1;k=1;while (getline < "file2" ){a[i++]=$2}}
{if($1 ~ /^[A-Z]/){l=length($0);for(j=1;j<=l;j=j+6){val=substr($0,j,6);if(val ~ /^[A-Z]/){val=substr($0,j,6);f1=1;printf val}else{if(f1==1){s=substr($0,j,8);j=j+2;gsub(" ","",s);f1=0;sm=s+a[k];k++;printf "%8.2f",sm}else{printf val}}};printf "\n"}
else{print}}' file1

This User Gave Thanks to raj_saini20 For This Post:
miriammiriam
# 25  
Old 07-25-2012
Quote:
Originally Posted by raj_saini20
try this
Code:
$awk 'BEGIN{f1=0;i=1;k=1;while (getline < "file2" ){a[i++]=$2}}
{if($1 ~ /^[A-Z]/){l=length($0);for(j=1;j<=l;j=j+6){val=substr($0,j,6);if(val ~ /^[A-Z]/){val=substr($0,j,6);f1=1;printf val}else{if(f1==1){s=substr($0,j,8);j=j+2;gsub(" ","",s);f1=0;sm=s+a[k];k++;printf "%8.2f",sm}else{printf val}}};printf "\n"}
else{print}}' file1

It is working again perfectly, I also changed the format of the of the 1st file as the previous one (another way to solve the problem Smilie ), But changing it might effect other parts of the code which reads the file in a specific format, This way, thanks to your code, is much much better and safe so Thank you very much I ll try to understand the code

REGARDS!! Smilie
# 26  
Old 07-25-2012
The same code reformatted via gawk --profile gives...
Code:
        # gawk profile, created Wed Jul 25 09:58:45 2012

        # BEGIN block(s)

        BEGIN {
                f1 = 0
                i = 1
                k = 1
                while (getline < "file2") {
                        a[i++] = $2
                }
        }

        # Rule(s)

        {
                if ($1 ~ /^[A-Z]/) {
                        l = length($0)
                        for (j = 1; j <= l; j = j + 6) {
                                val = substr($0, j, 6)
                                if (val ~ /^[A-Z]/) {
                                        val = substr($0, j, 6)
                                        f1 = 1
                                        printf val
                                } else {
                                        if (f1 == 1) {
                                                s = substr($0, j, 8)
                                                j = j + 2
                                                gsub(" ", "", s)
                                                f1 = 0
                                                sm = s + a[k]
                                                k++
                                                printf "%8.2f", sm
                                        } else {
                                                printf val
                                        }
                                }
                        }
                        printf "\n"
                } else {
                        print $0
                }
        }

...which might be easier to understand.
These 2 Users Gave Thanks to Ygor For This Post:
alister miriammiriam
# 27  
Old 07-25-2012
Quote:
Originally Posted by Ygor
The same code reformatted via gawk --profile gives...
Code:
        # gawk profile, created Wed Jul 25 09:58:45 2012

        # BEGIN block(s)

        BEGIN {
                f1 = 0
                i = 1
                k = 1
                while (getline < "file2") {
                        a[i++] = $2
                }
        }

        # Rule(s)

        {
                if ($1 ~ /^[A-Z]/) {
                        l = length($0)
                        for (j = 1; j <= l; j = j + 6) {
                                val = substr($0, j, 6)
                                if (val ~ /^[A-Z]/) {
                                        val = substr($0, j, 6)
                                        f1 = 1
                                        printf val
                                } else {
                                        if (f1 == 1) {
                                                s = substr($0, j, 8)
                                                j = j + 2
                                                gsub(" ", "", s)
                                                f1 = 0
                                                sm = s + a[k]
                                                k++
                                                printf "%8.2f", sm
                                        } else {
                                                printf val
                                        }
                                }
                        }
                        printf "\n"
                } else {
                        print $0
                }
        }

...which might be easier to understand.
It is easier to see the lines now, I see ok I ll try this one thank youu
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Order of data in Spool File

Hello, I have a shell script through which I am executing .sql file and spooling the result of Query from .sql . I want to spool the result in ascending order. Is there any parameter to be set to print result in ascending or descending order. Thanks in advance. (4 Replies)
Discussion started by: Aparna.N
4 Replies

2. Shell Programming and Scripting

How order a data matrix using awk?

is it possible to order the following row clusters from ascending to descending. thanx in advance input 1 2 4 0 1 2 4 0 3 3 3 3 1 5 1 0 1 5 1 0 6 0 0 0 5 1 1 1... (4 Replies)
Discussion started by: quincyjones
4 Replies

3. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

4. Shell Programming and Scripting

how to extract data from numbered files using linux in the numerical order-

Hi experts, I have a list of files containing forces as the only number as follows. Force1.txt Force2.txt Force3.txt Force4.txt Force5.txt . . . . . . . . . Force100.txt I want to put all the data(only a number ) in these forces files in the file with the same order like 1,2,3 ..100 .... (2 Replies)
Discussion started by: hamnsan
2 Replies

5. Shell Programming and Scripting

Help with sort data based on descending order problem

Input file 9.99331e-13 8.98451e-65 9.98418e-34 7.98319e-08 365592 111669 74942.9 0 Desired output 365592 111669 74942.9 7.98319e-08 1.99331e-13 6.98418e-34 (2 Replies)
Discussion started by: perl_beginner
2 Replies

6. Shell Programming and Scripting

Generating Gaussian Distributed Random Numbers

I want to generate an awk function that generated a Gaussian distributed set of random numbers. I need to implement the thing below in awk. Rnd is just a uniform random number between 0 and 1 function rgaussian(r1, r2) { Do v1 = 2 * Rnd - 1 v2 = 2 * Rnd - 1 ... (0 Replies)
Discussion started by: kristinu
0 Replies

7. Shell Programming and Scripting

Awk: Data Insert

Hi guys, I'm having some difficulties in insert some details to the following contents. I need to insert "TEST" under MIR & a value "25" to the next line. So far, I am able to insert "TEST" by using awk to capture MIR as the identifier. However, I am having some difficulties in inserting "25"... (3 Replies)
Discussion started by: nantheless
3 Replies

8. Shell Programming and Scripting

How to insert data befor some field in a row of data depending up on values in row

Hi I need to do some thing like "find and insert before that " in a file which contains many records. This will be clear with the following example. The original data record should be some thing like this 60119827 RTMS_LOCATION_CDR INSTANT_POSITION_QUERY 1236574686123083rtmssrv7 ... (8 Replies)
Discussion started by: aemunathan
8 Replies

9. Solaris

Not able to insert data

Hi All, I enhanced a perl script that creates a table with xyz_yyyymm and insert data from xyz table before truncate xyz.I have tested it successfully in dev but when i ran it on production new table xyz_yyyymm created but did not insert any records from xyz.(No errors were thrown)The perl... (1 Reply)
Discussion started by: megh
1 Replies

10. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this: sec;src;dst;proto 421;10.10.10.1;10.10.10.2;tcp 426;10.10.10.3;10.10.10.4;udp 442;10.10.10.5;10.10.10.6;tcp sec;src;fac;dst;proto 521;10.10.10.1;ab;10.10.10.2;tcp... (3 Replies)
Discussion started by: eric4
3 Replies
Login or Register to Ask a Question