Replace missing row with 0 Post: 302954564

Sponsored Content

Top Forums Shell Programming and Scripting Replace missing row with 0 Post 302954564 by ernesto on Wednesday 9th of September 2015 04:12:24 AM

09-09-2015

Registered User

Hi Sir,

it's not. We are getting stats based on a table which has 00:00:00 to 23:59:00 logging. and i am getting the total for an hour.

Hope you could help me.

Cheers,

---------- Post updated at 03:11 AM ---------- Previous update was at 03:02 AM ----------

We are using a hadoop nosql as the DB

Please see code below. The code part is more of the function that creates the files.

Code:

function func_formater
{

   file=$1
   outfile=$2

   awk 'BEGIN{
                   OFS="\t";

             }
             {
                   sub(/\|/,X,$NF);
                   A[substr($2,1,2)]+=$NF
             }
        END  {
                for(i in A){
                                print i":00" OFS A[i]
                           }
             }
        ' OFS="\t" $file | sort -n > $outfile

}

function 24Hour
{
        STARTTIME=$1
        ENDTIME=$2

        handlerFile="24HourHandler.dat"

        comp="SELECT time, server, component, sum(a+b) as total FROM <table> where logtime >= '${STARTTIME}' and logtime < '${ENDTIME}'  GROUP BY time, server, component ORDER BY time, server, component ASC;"

		#run in hadoop db
        out=`ssh $user@$host "echo \"$comp\" > ${compFile};dbname run -file=${compFile} address=$host -port=$port"`

        echo $outHandler | cut -d'-' -f13- | sed -e 's/---//g' | xargs -n5 | head -n -1  > 24HourExtract.dat

        ids=$(cat 24HourExtract.dat | awk -F'|' '{print $2}' | sort -u | sed -e 's/\n//g')
        comp=$(cat 24HourExtract.dat | awk -F'|' '{print $3}' | sed -e 's/\n//g' | sort -u)

        for id in `echo $ids`
        do
                for component in `echo $comp`
                do
                        grep -i ${id} 24HourExtract.dat | grep -i ${component} > ${id}_${component}.dat

                        #Call formater function
                        func_formater {id}_${component}.dat ${id}_${component}_final.dat

                        echo "${component}" > ${id}_${component}_final1.DAT
                        cat ${id}_${component}_final.dat >> ${id}_${component}_final1.DAT

                done
        done
}


TIMEEND="2015-09-08 23:59:00.0"
TIMESTART="2015-09-08 00:00:00.0"

24Hour "${TIMESTART}" "${TIMEEND}"

---------- Post updated at 03:12 AM ---------- Previous update was at 03:11 AM ----------

we are using Linux RedHat and bash as the default SHELL

ernesto

View Public Profile for ernesto

Find all posts by ernesto

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace missing standard folders from home directories.

Hi, I want to develop a script to replace missing folders from home directories. These may have been deleted by the user. A standard home directory will have these folders in it and nothing else: Desktop, Documents, Downloads, Library, Movies, Music, Pictures, Public, Sites I also want to...

2. Shell Programming and Scripting

Find and replace duplicate column values in a row

I have file which as 12 columns and values like this 1,2,3,4,5 a,b,c,d,e b,c,a,e,f a,b,e,a,h if you see the first column has duplicate values, I need to identify (print it to console) the duplicate value (which is 'a') and also remove duplicate values like below. I could be in two...

3. Shell Programming and Scripting

Replace last row of a column in bash/awk/sed

Hi, I've got a file with 3 columns which ends like this: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 1400 I want to replace the last row of the last column with the value 0. So my new file will end: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 ...

4. Shell Programming and Scripting

replace row separated by tab

Dear users, I have this problem, this is the example: 123 (tab) A (tab) B (tab) C (tab) 456 where the (tab) is actually the \t delimiter. I need to replace the A B and C for D E and F, this is: 123 (tab) D (tab) E (tab) F (tab) 456 The thing is that my file is quite long and this...

5. Shell Programming and Scripting

Replace Second row with First row in a File

Hi, I need to replace first row to second row: Input: A JACK WAS_${HH}_JACK .... Output should be: JACK WAS_A_JACK .... I tried below code.. awk '{TEMP= (NR==1)}; {sub(/${HH}/$TEMP/)}'

6. Shell Programming and Scripting

In a row, replace negative sign and find minimum value among four columns

Hi Friends, I have an input file like this chr1 100 200 1 2 3 4 chr1 150 200 4 5 6 7 chr2 300 400 9 6 7 1 chr2 300 410 -10 21 -11 13 chr3 700 900 -21 -22 130 165 Now, my output file is chr1 100 200 1 chr1 150 200 4 chr2 300 400 1 chr2 300 410 10 chr3 700 900 21 Remove...

7. UNIX for Dummies Questions & Answers

I want to replace the row with zero value

I have this file 5 9 7 23 0 5 7 78 0 3 7 0 5 44 9 0 i want to remove the row with zero value but i want to replace it with the above + under over 2 like this (the second zero) (78+3)/2

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000...

9. Shell Programming and Scripting

Replace First Column and First Row Data

HI Guys, I just want to replace data for First Column and Row Cell(1,1) Input :- Hello A B C X 1 2 3 Y 4 5 6 Z 7 8 9 Output:- Byee A B C X 1 2 3 Y 4 5 6 Z 7 8 9 From Hello to Byee .....And The Each file have Different String.

10. Shell Programming and Scripting

Add Row from First Row (Split Row)

HI Guys, I have Below Input :- RepigA_hteis522 ReptCfiEtrBsCll_aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 RepigA ReptCfiEtrBsCll hteis522 aofe MSL04_MSL2_A25_1A 0 9 MSL04_MSL2_A25_1B 0 9 MSL04_MSL2_A25_1C 0 9 Split Data in two first row...

LEARN ABOUT SUNOS

asadmin-show-component-status

asadmin-show-component-status(1AS)				   User Commands				asadmin-show-component-status(1AS)

NAME

       asadmin-show-component-status, show-component-status - displays the status of the deployed component

SYNOPSIS

       show-component-status  --user  admin_user  [--password admin_password] [--host localhost] [--port 4848] [--secure|-s] [--passwordfile file-
       name] [--terse=false] [--echo=false] [--interactive] component_name

       show-component-status gets the status of the deployed component. The status is a string representation returned by the server. The possible
       status strings include: enabled or disabled. This command is supported in remote mode only.

OPTIONS

       --user		       authorized domain application server administrative username.

       --password	       password to administer the domain application server.

       --host		       machine name where the domain application server is running.

       --port		       port number of the domain application server listening for administration requests.

       --secure 	       if true, uses SSL/TLS to communicate with the domain application server.

       --passwordfile	       file containing the domain application server password.

       --terse		       indicates that any output data must be very concise, typically avoiding human-friendly sentences and favoring well-
			       formatted data for consumption by a script.

       --echo		       setting to true will echo the command line statement on the standard output.

       --interactive	       prompts you for the required options that are not already specified.

OPERANDS

       component_name	       name of the component to be listed.

       Example 1: Using show-component-status

       asadmin> show-component-status sampleApplication
       Status of sampleApplication is enabled

       Where: the status of the sampleApplication application is shown.

EXIT STATUS

       0		       command executed successfully

       1		       error in executing the command

       asadmin-list-components(1AS), asadmin-list-sub-components(1AS)

J2EE 1.4 SDK							    March 2004					asadmin-show-component-status(1AS)