Replace missing row with 0 Post: 302954564

Sponsored Content

Top Forums Shell Programming and Scripting Replace missing row with 0 Post 302954564 by ernesto on Wednesday 9th of September 2015 04:12:24 AM

09-09-2015

Registered User

Hi Sir,

it's not. We are getting stats based on a table which has 00:00:00 to 23:59:00 logging. and i am getting the total for an hour.

Hope you could help me.

Cheers,

---------- Post updated at 03:11 AM ---------- Previous update was at 03:02 AM ----------

We are using a hadoop nosql as the DB

Please see code below. The code part is more of the function that creates the files.

Code:

function func_formater
{

   file=$1
   outfile=$2

   awk 'BEGIN{
                   OFS="\t";

             }
             {
                   sub(/\|/,X,$NF);
                   A[substr($2,1,2)]+=$NF
             }
        END  {
                for(i in A){
                                print i":00" OFS A[i]
                           }
             }
        ' OFS="\t" $file | sort -n > $outfile

}

function 24Hour
{
        STARTTIME=$1
        ENDTIME=$2

        handlerFile="24HourHandler.dat"

        comp="SELECT time, server, component, sum(a+b) as total FROM <table> where logtime >= '${STARTTIME}' and logtime < '${ENDTIME}'  GROUP BY time, server, component ORDER BY time, server, component ASC;"

		#run in hadoop db
        out=`ssh $user@$host "echo \"$comp\" > ${compFile};dbname run -file=${compFile} address=$host -port=$port"`

        echo $outHandler | cut -d'-' -f13- | sed -e 's/---//g' | xargs -n5 | head -n -1  > 24HourExtract.dat

        ids=$(cat 24HourExtract.dat | awk -F'|' '{print $2}' | sort -u | sed -e 's/\n//g')
        comp=$(cat 24HourExtract.dat | awk -F'|' '{print $3}' | sed -e 's/\n//g' | sort -u)

        for id in `echo $ids`
        do
                for component in `echo $comp`
                do
                        grep -i ${id} 24HourExtract.dat | grep -i ${component} > ${id}_${component}.dat

                        #Call formater function
                        func_formater {id}_${component}.dat ${id}_${component}_final.dat

                        echo "${component}" > ${id}_${component}_final1.DAT
                        cat ${id}_${component}_final.dat >> ${id}_${component}_final1.DAT

                done
        done
}


TIMEEND="2015-09-08 23:59:00.0"
TIMESTART="2015-09-08 00:00:00.0"

24Hour "${TIMESTART}" "${TIMEEND}"

---------- Post updated at 03:12 AM ---------- Previous update was at 03:11 AM ----------

we are using Linux RedHat and bash as the default SHELL

ernesto

View Public Profile for ernesto

Find all posts by ernesto

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace missing standard folders from home directories.

Hi, I want to develop a script to replace missing folders from home directories. These may have been deleted by the user. A standard home directory will have these folders in it and nothing else: Desktop, Documents, Downloads, Library, Movies, Music, Pictures, Public, Sites I also want to...

2. Shell Programming and Scripting

Find and replace duplicate column values in a row

I have file which as 12 columns and values like this 1,2,3,4,5 a,b,c,d,e b,c,a,e,f a,b,e,a,h if you see the first column has duplicate values, I need to identify (print it to console) the duplicate value (which is 'a') and also remove duplicate values like below. I could be in two...

3. Shell Programming and Scripting

Replace last row of a column in bash/awk/sed

Hi, I've got a file with 3 columns which ends like this: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 1400 I want to replace the last row of the last column with the value 0. So my new file will end: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 ...

4. Shell Programming and Scripting

replace row separated by tab

Dear users, I have this problem, this is the example: 123 (tab) A (tab) B (tab) C (tab) 456 where the (tab) is actually the \t delimiter. I need to replace the A B and C for D E and F, this is: 123 (tab) D (tab) E (tab) F (tab) 456 The thing is that my file is quite long and this...

5. Shell Programming and Scripting

Replace Second row with First row in a File

Hi, I need to replace first row to second row: Input: A JACK WAS_${HH}_JACK .... Output should be: JACK WAS_A_JACK .... I tried below code.. awk '{TEMP= (NR==1)}; {sub(/${HH}/$TEMP/)}'

6. Shell Programming and Scripting

In a row, replace negative sign and find minimum value among four columns

Hi Friends, I have an input file like this chr1 100 200 1 2 3 4 chr1 150 200 4 5 6 7 chr2 300 400 9 6 7 1 chr2 300 410 -10 21 -11 13 chr3 700 900 -21 -22 130 165 Now, my output file is chr1 100 200 1 chr1 150 200 4 chr2 300 400 1 chr2 300 410 10 chr3 700 900 21 Remove...

7. UNIX for Dummies Questions & Answers

I want to replace the row with zero value

I have this file 5 9 7 23 0 5 7 78 0 3 7 0 5 44 9 0 i want to remove the row with zero value but i want to replace it with the above + under over 2 like this (the second zero) (78+3)/2

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000...

9. Shell Programming and Scripting

Replace First Column and First Row Data

HI Guys, I just want to replace data for First Column and Row Cell(1,1) Input :- Hello A B C X 1 2 3 Y 4 5 6 Z 7 8 9 Output:- Byee A B C X 1 2 3 Y 4 5 6 Z 7 8 9 From Hello to Byee .....And The Each file have Different String.

10. Shell Programming and Scripting

Add Row from First Row (Split Row)

HI Guys, I have Below Input :- RepigA_hteis522 ReptCfiEtrBsCll_aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 RepigA ReptCfiEtrBsCll hteis522 aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 Split Data in two first row...

LEARN ABOUT SUNOS

wsreg_set_parent

wsreg_set_parent(3WSREG)			    Product Install Registry Library Functions				  wsreg_set_parent(3WSREG)

NAME

       wsreg_set_parent, wsreg_get_parent - set or get the parent of a component

SYNOPSIS

       cc [flag ...]  file ...-lwsreg [library ...]
       #include <wsreg.h>

       void wsreg_set_parent(Wsreg_component *comp, const Wsreg_component *parent);

       Wsreg_component *wsreg_get_parent(const Wsreg_component *comp);

DESCRIPTION

       The wsreg_set_parent() function sets the parent specified by parent of the component specified by comp.

       The wsreg_get_parent() function gets the parent of the component specified by comp.

RETURN VALUES

       The wsreg_get_parent() function returns a pointer to a Wsreg_component structure that represents the parent of the specified component.	If
       the specified component does not have a parent, NULL is returned.  If a non-null value is  returned,  it  the  caller's	responsibility	to
       release	the  memory  associated  with the resulting Wsreg_component pointer with a call to wsreg_free_component(). See wsreg_create_compo-
       nent(3WSREG).

USAGE

       The parent of a component is set as a result of registering the parent component.  When a component that has children is registered, all of
       the child components are updated to reflect the newly registered component as their parent.  This update only occurs if the child component
       does not already have a parent component set.

       The specified parent component is reduced to a lightweight component reference that uniquely identifies the parent in the  product  install
       registry.  This lightweight reference includes the parent's uuid and instance number.

       The parent must be registered before a call to wsreg_set_parent() can be made, since the parent's instance number must be known at the time
       the wsreg_set_parent() function is called.

       A process needing to call wsreg_set_parent() or wsreg_get_parent() must have read access to the product install registry.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |MT-Level		     |Unsafe			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       wsreg_can_access_registry(3WSREG),	 wsreg_create_component(3WSREG),	 wsreg_initialize(3WSREG),	   wsreg_register(3WSREG),
       wsreg_set_instance(3WSREG), attributes(5)

SunOS 5.10							    22 Sep 2000 					  wsreg_set_parent(3WSREG)