Sponsored Content
Top Forums Shell Programming and Scripting SMA (Single Moving Average) and Standard Deviation Post 303003085 by RudiC on Thursday 7th of September 2017 12:10:32 PM
Old 09-07-2017
Try
Code:
awk -v points=5 '

NR == 1 {next
        }

        {a[(NR-1) % points] = $2
         if ((NR-1)>=points)    {total_a = 0
                                 for (i = 0; i < points; ++i)   {total_a = total_a + a[i]
#                                                                print i, a[i], total_a, total_a / 5 
                                                                }
                                 AVG = total_a/points
                                 ERR = 0
                                 for (i = 0; i < points; ++i)   {ERR += (AVG - a[i]) * (AVG - a[i])
                                                                }
                                 print $1, $2, AVG, sqrt(ERR/points)
                                }  
        }
' OFS="	" file1
5	10.1	10.096	0.0422374
6	10.06	10.09	0.0447214
7	10.15	10.092	0.0470744
8	9.91	10.07	0.085557
9	9.99	10.042	0.0842378
10	9.72	9.966	0.146233
11	9.75	9.904	0.158442
12	9.57	9.788	0.147838
13	9.62	9.73	0.145465
14	9.97	9.726	0.138362
15	9.71	9.724	0.138506
16	9.77	9.728	0.139485
17	9.88	9.79	0.12345
18	9.71	9.808	0.102059
19	9.75	9.764	0.062482
20	9.53	9.728	0.113912
21	9.46	9.666	0.152132
22	9.38	9.566	0.142632
23	9.63	9.55	0.12946
24	9.67	9.534	0.106696
25	9.92	9.612	0.187339
26	10.04	9.728	0.231638

This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script for finding standard deviation

I have a CSV file that looks like 0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0 10,11,7,0,4,12,2,3,7,0,11,3,12,4,0,5,5,4,5,0,8,6,12,0,9,3,3,0,2,7,8 19,11,7,0,4,14,16,10,8,2,13,7,15,6,0,76,6,4,10,0,18,10,17,1,11,3,3,0,9,9,8... (7 Replies)
Discussion started by: RJ17
7 Replies

2. Shell Programming and Scripting

Mean and Standard deviation

Hi all, I am new to shell scripting and wanna calculate the mean and standard deviation using shell programming. I have a file with letters that are repeating and their corresponding duration a 0.32 a 0.89 aa 0.34 aa 0.23 au 0.012 au 0.26... (4 Replies)
Discussion started by: lakshmikanth.pg
4 Replies

3. UNIX for Dummies Questions & Answers

Calculating the Standard Deviation for a column

Hi all, I want to calculate the standard deviation for a column (happens to be column 3). Does any know of simple awk script to do this? Thanks (1 Reply)
Discussion started by: kylle345
1 Replies

4. Shell Programming and Scripting

using awk to print average and standard deviation into a file

Hi I want to use awk to print avg and st deviation but it does not go into a file for column 1 only. I can do average and # of records but i cannot get st deviation. awk '{sum+=$1} END { print "Average = ",sum/NR}' thanks (1 Reply)
Discussion started by: phil_heath
1 Replies

5. Shell Programming and Scripting

Moving Averages SMA !

Hello, I am trying to create a awk script to calculate the simple moving average of two fields but I got only the result of the first field. Could you please help me to fix this awk shell script awk -F, -v points=5 ' { a = $2; b = $3; if(NR>=points) { total_a = 0; total_b... (1 Reply)
Discussion started by: csierra
1 Replies

6. Shell Programming and Scripting

Standard deviation in awk

Hi all, I need to find the standard deviation of each column of a dataset below for each hour. The data is given in 5 second intervals as shown below DATE TIME FRAC_DAYS_SINCE_JAN1 FRAC_HRS_SINCE_JAN1 EPOCH_TIME ... (11 Replies)
Discussion started by: gd9629
11 Replies

7. Shell Programming and Scripting

AWK script for standard deviation / root mean square deviation

I have a file with say 50 columns, each containing a whole lot of data. Each column contains data from a separate simulation, but each simulation is related to the data in the last (REFERENCE) column $50 I need to calculate the RMS deviation for each data line, i.e. column 1 relative to... (12 Replies)
Discussion started by: chrisjorg
12 Replies

8. Shell Programming and Scripting

calculating row-wise standard deviation using awk

Hi, I have a file containing 100,000 rows-by-120 columns and I need to compute for the standard deviation for each row. Any idea on how to calculate row-wise standard deviation using awk? My sample data looks like this: input data: 23 35 12 25 16 17 18 19 29 12 12 26 15 14 15 23 12 12... (2 Replies)
Discussion started by: ida1215
2 Replies

9. Shell Programming and Scripting

Computing average and standard deviation from multiple text files

Hello there, I found an elegant solution to computing average values from multiple text files awk '{for (i=1;i<=NF;i++){if ($i!~"n/a"){a+=$i}else{b++}}}END{for (i=1;i<=FNR;i++){for (j=1;j<=NF;j++){printf (a/(3-b))((b>0)?"~"b" ":" ")};printf "\n"}}' file1 file2 file3 I tried to modify... (2 Replies)
Discussion started by: charmmilein
2 Replies

10. Shell Programming and Scripting

Output mean and standard deviation of a row

I have a file that looks that this: 820 890 530 1650 1600 1800 1850 1900 2270 1640 2300 1670 2080 2200 2350 1150 1630 2210 I would like to output the mean and standard deviation of each row so that my final output would look like this 820 890 530 746.667 155.849 1650 1600 1800... (5 Replies)
Discussion started by: kayak
5 Replies
masfcnv(1M)						  System Administration Commands					       masfcnv(1M)

NAME
masfcnv - SNMP configuration migration script SYNOPSIS
/usr/sfw/lib/sma_snmp/masfcnv [-cimnrs] [-l agentmaster] [-p enabledisableerror] [-t noneadd] [-u agentmastererror] [-y agentmaster- error] masfcnv [-V] masfcnv [-?] DESCRIPTION
The masfcnv script is used to assist the system administrator in migrating an existing set of configuration files for the Sun SNMP Manage- ment Agent for Sun Fire and Netra Systems (MASF) to the Systems Management Agent (SMA). The script accepts as input the currently installed set of MASF and SMA configuration files and outputs a new set of SMA configuration files. Existing SMA configuration files are backed up by appending .bak to the filename. The administrator can choose to output the new configuration to standard output, instead of replacing the current configuration, by specifying the -n option. The migration script must be run as the superuser. Failure to do so causes the script to exit with an error message. Before running the script you should ensure that both the SMA and MASF agents are not running. If the agents are running they will be shut down by the script. The migration script installs a new startup script for the MASF agent in /etc/init.d, as well as a backup of the old script. During migra- tion, MASF will be configured as an AgentX subagent of SMA. All migration settings will be migrated to the SMA configuration file. The migration script aborts if any unrecognized directives are found in either the MASF configuration files or the SMA configuration files. This can be overridden with the -i option. If this option is selected, the behavior is to retain unrecognized directives that were present in the SMA configuration, but remove those present in the MASF configuration. The migration script then proceeds to migrate access control and trap configuration. As a side effect of running the migration script, the following directives might be expanded by the script into multiple directives with an equivalent interpretation: o rwcommunity o rocommunity o rwuser o rouser o trapcommunity o trapsink o trap2sink o informsink Access Control Migration Access control directives are expanded into the equivalent com2sec, group, access and view directives. Existing group names are renamed by prepending a prefix to avoid conflict with any which may already be defined in SMA. When migrating SNMPv1 or v2c access control, a conflict can occur if both MASF and SMA configuration files have defined access permissions for the same community and source address. The default behavior is to abort with a message, unless a use of the -y option specifies other- wise. If -y agent is specified then the MASF configuration takes precedence. If -y master is specified then the SMA configuration is retained. When migrating USM configuration (SNMPv3), a conflict can occur if both SMA and MASF configurations define a user with the same security- Name. If this occurs, the behavior of the script is determined by the -u option. If -u agent is specified, the configuration of the user defined in the MASF configuration files is the one that is retained. Otherwise, if the -u master option is specified, the use defined in the SMA configuration files is retained. By default, the migration script attempts to migrate USM users from MASF to SMA. The script determines whether there are any SNMPv3 users present in the SMA configuration and whether the default engineID has been overridden in the SMA configuration files. If neither of these conditions obtain, then the any usmUser statements containing localized authentication keys can be migrated to SMA, along with the MASF engineID. This results in the engineID of the SMA master agent changing. If the script determines that there are existing SNMPv3 users or a manually configured engineID present in the SMA configuration, only those users defined in createUser statements are transferred. Those users that were defined in usmUser statements are transferred but will have their passwords reset to a random value. You should notify your users of their new password or reset the password yourself by editing the newly-generated configuration file. Trap/Inform Migration The migration script performs a check to determine whether a trap destination defined for MASF is already specified in an existing SMA trapsink, trap2sink or informsink directive. If this is the case, then the directive in the MASF configuration will be discarded to avoid duplicate traps/informs being received. trapsink, trap2sink and informsink directives specified in the existing SMA configuration are considered valid destinations for MASF traps/informs and will receive them from the MASF subagent after migration. If the -t none option was specified on the command line, the migration script carries over any remaining MASF trap/inform directives with- out modification. If the -t add option was specified (the default), the migration script expands any trapsink, trap2sink, or informsink directives to use the TARGET-MIB and NOTIFICATION-MIB. The TARGET-MIB specifies targets using IP addresses, so it might be desirable to use the -t none option if, for example, the network allocates IP addresses to hostnames dynamically by means of DHCP. The expanded directives defines filters specific to the MASF agent so that traps from other subagents will not be received by migrated trap destinations. Existing filters present in the SMA configuration are, by default, not modified and might or might not receive MASF traps, depending upon the filters that were originally defined for them. If the -l option is specified, any filters already defined in the TARGET-MIB and the NOTIFICATION-MIB for SMA are extended to include traps from MASF. In the event that a trap destination is already configured in the TARGET-MIB with the same target address and community as an existing MASF trap/inform sink, a conflict will arise. If -l agent was specified and a conflict arises, the migration script uses the target SNMP parameters (that is, the SNMP version and choice of trap/inform) defined by the MASF trap/informsink directive to send traps to this destination. Otherwise, if the -l master option was specified, the conflict will be resolved using the target SNMP parameters specified in the SMA configuration. Miscellaneous If the migration script encounters in the MASF configuration file any of the directives listed below and the directives are either not present or differ from the SMA configuration, the script will log a warning message. o syslocation o syscontact o sysname o sysservices o agentgroup o agentuser o authtrapenable OPTIONS
The following options are supported: -? --help Displays usage information. -c --no-community Do not transfer v1/v2c communities. -i --ignore-unrecognized-directives Continue processing if unrecognized directives are present. -l agent | master --master-trap-target=agent | master If agent is specified, the existing SMA trap targets will be configured to receive traps that were previously sent to destinations for the Sun Fire SNMP agent. If master is specified, the targets will be configured to receive Sun Fire SNMP traps, but existing SNMP tar- get parameters will be used. -m --no-usmuser Do not transfer usm (v3) users. -n --dry-run Run the migration without modifying any files. If an error arises, continue processing. This can be used to determine the likely migra- tion issues. -p enable | disable | error --use-agent-port=enable | disable | error Indicates whether the port originally used by the Sun Fire SNMP agent should be used by the SMA agent after migration (if the two agents are using different ports). If enable is specified, then the port used by the Sun Fire SNMP agent will also be used by the SMA agent after migration. If disable is specified, the ports used by SMA will not be updated by the migration tool. If the error option is specified and the SMA agent is not already using the same ports as those used by the original Sun Fire SNMP agent, an error is reported and the migration process is terminated. If no option is specified the default behavior is equivalent to the error flag. -r --no-trap Do not transfer trap destinations. -s --skip-user If a user is found in the MASF configuration file that cannot be created in the new configuration because of a change in the engine ID, then output a message indicating that the user could not be migrated (needs to be manually recreated) and continue processing. If this option is not present, the migration tool will consider such a situation as an error and abort. -t none | add --trap-filter=none | add If none is specified then the script will copy trap directives directly. The administrator might need to manually update the configura- tion file to ensure traps are only delivered to their intended destinations. If add is specifed, trap filters will be constructed so that traps originating from the original Sun Fire SNMP agent are delivered only to the destinations that originally received them. The default behavior is add. -u agent | master | error --select-user=agent | master | error Specifies that if a user with the same name is found in both configuration files that the conflict is to be resolved using the speci- fied configuration file as input. Selecting a user from a particular will also cause the group declaration for that user to be taken from the same file. If agent is specified then the user will be taken from the configuration file for the Sun Fire SNMP Agent. If mas- ter is specified, the user will be taken from the SMA configuration. Otherwise, if error is given, the script will terminate. If this option is not present, the default behavior is equivalent to the error flag. -V --version Display the version of this script. -y agent | master | error --select-community=agent | master | error EXAMPLES
Example 1: Simplest Case The command shown below is appropriate for a simple migration. The migration fails if there are any potential conflicts. # masfcnv Example 2: Migrating Such That MASF Settings Override To migrate the MASF configuration such that it will always succeed, that MASF settings will override in the event of a conflict with SMA, and that access will still be provided on the original MASF port, enter: # masfcnv -is -l agent -p enable -u agent -y agent Example 3: Dry Run, Retaining SMA Settings To attempt a dry run and migrate the configuration such that any conflicts will be resolved by retaining existing SMA settings, enter: masfcnv -l master -u master -y master EXIT STATUS
0 Success. non-zero A problem occurred during migration. FILES
/etc/sma/snmp/snmpd.conf /var/sma_snmp/snmpd.conf SMA configuration files /etc/opt/SUNWmasf/conf/snmpd.conf /var/opt/SUNWmasf/snmpd.dat MASF configuration files /tmp/sma_migration.log masfcnv log file ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWsmcmd | +-----------------------------+-----------------------------+ |Interface Stability |Stable | +-----------------------------+-----------------------------+ SEE ALSO
attributes(5) SunOS 5.10 15 Jan 2004 masfcnv(1M)
All times are GMT -4. The time now is 11:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy