Replace missing row with 0 Post: 302954564

Sponsored Content

Top Forums Shell Programming and Scripting Replace missing row with 0 Post 302954564 by ernesto on Wednesday 9th of September 2015 04:12:24 AM

09-09-2015

Registered User

Hi Sir,

it's not. We are getting stats based on a table which has 00:00:00 to 23:59:00 logging. and i am getting the total for an hour.

Hope you could help me.

Cheers,

---------- Post updated at 03:11 AM ---------- Previous update was at 03:02 AM ----------

We are using a hadoop nosql as the DB

Please see code below. The code part is more of the function that creates the files.

Code:

function func_formater
{

   file=$1
   outfile=$2

   awk 'BEGIN{
                   OFS="\t";

             }
             {
                   sub(/\|/,X,$NF);
                   A[substr($2,1,2)]+=$NF
             }
        END  {
                for(i in A){
                                print i":00" OFS A[i]
                           }
             }
        ' OFS="\t" $file | sort -n > $outfile

}

function 24Hour
{
        STARTTIME=$1
        ENDTIME=$2

        handlerFile="24HourHandler.dat"

        comp="SELECT time, server, component, sum(a+b) as total FROM <table> where logtime >= '${STARTTIME}' and logtime < '${ENDTIME}'  GROUP BY time, server, component ORDER BY time, server, component ASC;"

		#run in hadoop db
        out=`ssh $user@$host "echo \"$comp\" > ${compFile};dbname run -file=${compFile} address=$host -port=$port"`

        echo $outHandler | cut -d'-' -f13- | sed -e 's/---//g' | xargs -n5 | head -n -1  > 24HourExtract.dat

        ids=$(cat 24HourExtract.dat | awk -F'|' '{print $2}' | sort -u | sed -e 's/\n//g')
        comp=$(cat 24HourExtract.dat | awk -F'|' '{print $3}' | sed -e 's/\n//g' | sort -u)

        for id in `echo $ids`
        do
                for component in `echo $comp`
                do
                        grep -i ${id} 24HourExtract.dat | grep -i ${component} > ${id}_${component}.dat

                        #Call formater function
                        func_formater {id}_${component}.dat ${id}_${component}_final.dat

                        echo "${component}" > ${id}_${component}_final1.DAT
                        cat ${id}_${component}_final.dat >> ${id}_${component}_final1.DAT

                done
        done
}


TIMEEND="2015-09-08 23:59:00.0"
TIMESTART="2015-09-08 00:00:00.0"

24Hour "${TIMESTART}" "${TIMEEND}"

---------- Post updated at 03:12 AM ---------- Previous update was at 03:11 AM ----------

we are using Linux RedHat and bash as the default SHELL

ernesto

View Public Profile for ernesto

Find all posts by ernesto

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace missing standard folders from home directories.

Hi, I want to develop a script to replace missing folders from home directories. These may have been deleted by the user. A standard home directory will have these folders in it and nothing else: Desktop, Documents, Downloads, Library, Movies, Music, Pictures, Public, Sites I also want to...

2. Shell Programming and Scripting

Find and replace duplicate column values in a row

I have file which as 12 columns and values like this 1,2,3,4,5 a,b,c,d,e b,c,a,e,f a,b,e,a,h if you see the first column has duplicate values, I need to identify (print it to console) the duplicate value (which is 'a') and also remove duplicate values like below. I could be in two...

3. Shell Programming and Scripting

Replace last row of a column in bash/awk/sed

Hi, I've got a file with 3 columns which ends like this: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 1400 I want to replace the last row of the last column with the value 0. So my new file will end: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 ...

4. Shell Programming and Scripting

replace row separated by tab

Dear users, I have this problem, this is the example: 123 (tab) A (tab) B (tab) C (tab) 456 where the (tab) is actually the \t delimiter. I need to replace the A B and C for D E and F, this is: 123 (tab) D (tab) E (tab) F (tab) 456 The thing is that my file is quite long and this...

5. Shell Programming and Scripting

Replace Second row with First row in a File

Hi, I need to replace first row to second row: Input: A JACK WAS_${HH}_JACK .... Output should be: JACK WAS_A_JACK .... I tried below code.. awk '{TEMP= (NR==1)}; {sub(/${HH}/$TEMP/)}'

6. Shell Programming and Scripting

In a row, replace negative sign and find minimum value among four columns

Hi Friends, I have an input file like this chr1 100 200 1 2 3 4 chr1 150 200 4 5 6 7 chr2 300 400 9 6 7 1 chr2 300 410 -10 21 -11 13 chr3 700 900 -21 -22 130 165 Now, my output file is chr1 100 200 1 chr1 150 200 4 chr2 300 400 1 chr2 300 410 10 chr3 700 900 21 Remove...

7. UNIX for Dummies Questions & Answers

I want to replace the row with zero value

I have this file 5 9 7 23 0 5 7 78 0 3 7 0 5 44 9 0 i want to remove the row with zero value but i want to replace it with the above + under over 2 like this (the second zero) (78+3)/2

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000...

9. Shell Programming and Scripting

Replace First Column and First Row Data

HI Guys, I just want to replace data for First Column and Row Cell(1,1) Input :- Hello A B C X 1 2 3 Y 4 5 6 Z 7 8 9 Output:- Byee A B C X 1 2 3 Y 4 5 6 Z 7 8 9 From Hello to Byee .....And The Each file have Different String.

10. Shell Programming and Scripting

Add Row from First Row (Split Row)

HI Guys, I have Below Input :- RepigA_hteis522 ReptCfiEtrBsCll_aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 RepigA ReptCfiEtrBsCll hteis522 aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 Split Data in two first row...

LEARN ABOUT SUNOS

wsreg_get_type

wsreg_set_type(3WSREG)				    Product Install Registry Library Functions				    wsreg_set_type(3WSREG)

NAME

       wsreg_set_type, wsreg_get_type - set or get the type of a component

SYNOPSIS

       cc [ flag... ]  file ... -lwsreg [ library ... ]
       #include <wsreg.h>

       int wsreg_set_type(Wsreg_component *comp, Wsreg_component_type type);

       Wsreg_component_type wsreg_get_type(const Wsreg_component *comp);

DESCRIPTION

       The wsreg_set_type() function sets the type specified by type in the component specified by comp.

       The wsreg_get_type() function retrieves the type from the component specified by comp.

RETURN VALUES

       The wsreg_set_type() function returns a non-zero value if the type is set successfully; otherwise 0 is returned.

       The wsreg_get_type() function returns the type currently set in the component specified by comp.

USAGE

       The  component type is used to indicate whether a Wsreg_component structure represents a product, feature, or component.  The type argument
       can be one of the following:

       WSREG_PRODUCT	       Indicates the Wsreg_component represents a product.  A product is a collection of features and/or components.

       WSREG_FEATURE	       Indicates the Wsreg_component represents a feature.  A feature is a collection of components.

       WSREG_COMPONENT	       Indicates the Wsreg_component represents a component.  A component is a collection of files that may be installed.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |MT-Level		     |Unsafe			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       wsreg_create_component(3WSREG), wsreg_initialize(3WSREG), wsreg_register(3WSREG), wsreg_set_instance(3WSREG), attributes(5)

SunOS 5.10							    22 Sep 2000 					    wsreg_set_type(3WSREG)