Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Transpose Datefield from rows to column + Print time diff

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transpose Datefield from rows to column + Print time diff
# 1  
Old 10-19-2012
Transpose Datefield from rows to column + Print time diff
Hi Experts,

Can you please help me in transposing Datefield from rows to column and calculate the time difference for each of the Jobids:

Input File:
Code:
08/23/2012 12:36:09,JOB_5340
08/23/2012 12:36:14,JOB_5340
08/23/2012 12:36:22,JOB_5350
08/23/2012 12:36:26,JOB_5350

Required Output:
Code:
08/23/2012 12:36:14,08/23/2012 12:36:09,00:00:05,JOB_5340
08/23/2012 12:36:26,08/23/2012 12:36:22,00:00:04,JOB_5350

Many Thanks
# 2  
Old 10-20-2012
This is one way:
Code:
$ cat t
08/23/2012 12:36:09,JOB_5340
08/23/2012 12:36:14,JOB_5340
08/23/2012 12:36:22,JOB_5350
08/23/2012 12:36:26,JOB_5350

$ cat test.sh
while IFS="," read inDate jobName
do
  (( c+=1 ))
  if [[ $c -eq 1 ]]; then
    ts1=$inDate
  else
    s=$(( $(date -d "$inDate" +%s) - $(date -d "$ts1" +%s) ))
    printf "$inDate,$ts1,%02d:%02d:%02d" $((s/3600)) $((s%3600/60)) $((s%60))
    echo ",$jobName"
    c=0
  fi
done <t

$ test.sh
08/23/2012 12:36:14,08/23/2012 12:36:09,00:00:05,JOB_5340
08/23/2012 12:36:26,08/23/2012 12:36:22,00:00:04,JOB_5350

This User Gave Thanks to spacebar For This Post:
asnandhakumar
# 3  
Old 10-22-2012
Hi Spacebar,

Thanks a lot for your quick response. I have tried the code and this throws some errors:

./trans.ksh[17]: c+=1 : 0403-009 The specified number is not valid for this command.

Can you also please explain the given code

Thanks,
Nandha
# 4  
Old 10-22-2012
Quote:
Originally Posted by asnandhakumar
./trans.ksh[17]: c+=1 : 0403-009 The specified number is not valid for this command.

Can you also please explain the given code

Thanks,
Nandha

The attempt is to add 1 to the variable c each time through the loop. If unset, the shell should assume 0. I'm guessing your version of Kshell is old enough that it doesn't support it. You could try replacing the line with


Code:
c=$(( $c  + 1 ))

and add
Code:
c=0

before the loop.


If it still isn't working, please post the shell, and version, that you are using in addition to the errors.
# 5  
Old 10-22-2012
Add this statement before the 'when' statement and test:
Code:
c=0

# 6  
Old 10-22-2012
Spacebar, Agama

I have tried the suggested changes, and it still errors out...

Changed code:
Code:
while IFS="," read inDate jobName
c=0
do
c=$(($c+1))
if [[$c -eq 1]]; then
ts1=$inDate
else
s=$(($(date -d "$inDate" +%s) - $(date -d "$ts1" +%s)))
printf "$inDate,$ts1,%02d:%02d:%02d" $((s/3600)) $((s%3600/60)) $((s%60))
echo ",$jobName"
fi
done <t.csv

Error Message:
Code:
./trans.ksh[19]: [[1:  not found.
date: Not a recognized flag: d
Usage: date [-u] [+"Field Descriptors"]
date: Not a recognized flag: d
Usage: date [-u] [+"Field Descriptors"]
./trans.ksh[22]:  - : 0403-053 Expression is not complete; more tokens expected.

I'm using ksh on AIX machine, and the ksh version is Version M-11/16/88f

Many thanks for your help
NK
# 7  
Old 10-22-2012
Since your using ksh88, This is another way you can get what you want:
Code:
Perl script:
#!/usr/bin/perl -w
use POSIX;
use strict;
use warnings;

my $rc;             # Read count
my @r;              # Record fields
my @ts;             # Time stamp fields
my @sts;            # Save of start timestamp
my ( $sec, $min, $hour, $day, $mon, $year, $wday, $yday );
my $ut_str_time;    # Start timestamp in unix time
my $ut_end_time;    # End   timestamp in unix time
my $diff;           # Difference in seconds
my $dstr;           # Difference in format: hh:mm:ss

open ( INFILE, "<t") or die ("***Error- Couldn't open file: t, $!\n");
while ( <INFILE> ) {
  $rc++;
  chomp;
  @r  = split((','),$_);
  @ts = split((':|/| '),$r[0]);
  if ( $rc == 1 ) {
    @sts = @ts;
  } else {
    $ut_end_time = mktime( $ts[5], $ts[4], $ts[3], $ts[1], ($ts[0]-1), ($ts[2]-1900), 0, 0 );
    $ut_str_time = mktime( $sts[5], $sts[4], $sts[3], $sts[1], ($sts[0]-1), ($sts[2]-1900), 0, 0 );
    $diff        = $ut_end_time - $ut_str_time;
    $dstr        = POSIX::strftime("%H:%M:%S",$diff,0,0,0,0,0,0,0,0);
    print "$ts[0]/$ts[1]/$ts[2] $ts[3]:$ts[5]:$ts[4],$sts[0]/$sts[1]/$sts[2] $sts[3]:$sts[5]:$sts[4],$dstr,$r[1]\n";
    $rc = 0;
  }
}

$ cat t
08/23/2012 12:36:09,JOB_5340
08/23/2012 12:36:14,JOB_5340
08/23/2012 12:36:22,JOB_5350
08/23/2012 12:36:26,JOB_5350

$ test.pl
08/23/2012 12:14:36,08/23/2012 12:09:36,00:00:05,JOB_5340
08/23/2012 12:26:36,08/23/2012 12:22:36,00:00:04,JOB_5350

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to transpose pieces of data in a column to multiple rows?

Hello Everyone, I am very new to the world of regular expressions. I am trying to use grep/sed for the following: Input file is something like this and there are multiple such files: abc 1 2 3 4 5 ***END*** abc 6 7 8 9 ***END*** abc 10 (2 Replies)
Discussion started by: shellnewuser
2 Replies

2. Shell Programming and Scripting

Linux - Transpose rows into column

hello, I have a server that collect some performance statistics of 4 servers in the following input file : $ cat inputfile Time,A,Server1,KPI1,data1 Time,A,Server1,KPI2,data2 Time,A,Server1,KPI3,data3 Time,A,Server1,KPI4,data4 Time,A,Server1,KPI5,data5 Time,A,Server2,KPI1,data6... (9 Replies)
Discussion started by: capitain25
9 Replies

3. Programming

To transpose rows to column in hadoop

Hi, i am having an HDFS file which is comma seperated, i need to transpose from rows to column only the header columns text.csv cnt,name,place 1,hi,nz 2,hello,aus I need cnt, name, place while using below command in hadoop getting the error hadoop fs -fmt -1 text.csv (0 Replies)
Discussion started by: rohit_shinez
0 Replies

4. Shell Programming and Scripting

Peel syntax for transpose rows into column

Dear all, Plz let me know syntax for transposing rows into column in perl, I am having 30 csv files which are merged into a single xls sheet. but i want to transpose each row into column in excel sheet in each tab (1 CSV = 1tab in xls sheet) example is as below ... (0 Replies)
Discussion started by: sagar_1986
0 Replies

5. Shell Programming and Scripting

Transpose multiple rows (with a mix of space and enter) to a single column

How to change the uploaded weekly file data to the following format? New Well_Id,Old Well_Id,District,Thana,Date,Data,R.L,WellType,Lati.,Longi. BAG001,PT006,BARGUNA,AMTALI,1/2/1978,1.81,2.29,Piezometer,220825,901430 BAG001,PT006,BARGUNA,AMTALI,1/9/1978,1.87,2.29,Piezometer,220825,901430... (3 Replies)
Discussion started by: sara.nowreen
3 Replies

6. Shell Programming and Scripting

Transpose Column of Data to Rows

I can no longer find my commands, but I use to be able to transpose data with common fields from a single column to rows using a command line. My data is separated as follows: NAME=BOB ADDRESS=COLORADO PET=CAT NAME=SUSAN ADDRESS=TEXAS PET=BIRD NAME=TOM ADDRESS=UTAH PET=DOG I would... (7 Replies)
Discussion started by: docdave78
7 Replies

7. Shell Programming and Scripting

awk to transpose preceding row to 1st column of next rows

Gurus: How can I transpose the output below to a format in which I can plot a graph to show VSZ memory usage by PIDs? stdout: Tue Jan 22 07:29:19 CUT 2013 42336296 1841272 java wilyadm 21889232 438616 jlaunch sidadm 42532994 414336 jlaunch sidadm Tue Jan 22 07:49:20 CUT 2013... (1 Reply)
Discussion started by: ux4me
1 Replies

8. Shell Programming and Scripting

Transpose timestamp based on column values and calculate time difference

Hello Expert, I need to transpose Date-Timestamp based on same column values and calculate time difference. The input file would be as below and required output is mentioned in the bottom INPUT File ======== 08/23/2012 12:36:09 JOB_5340 08/23/2012 12:36:14 JOB_5340 08/23/2012... (2 Replies)
Discussion started by: asnandhakumar
2 Replies

9. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

10. Shell Programming and Scripting

awk transpose rows to column

Need to transpose in awk rows to column like this: input: A1,6,5,4 3,2,1, A2,8,7,9,10,11,12,13,14 A3,1,2,3,5,7,8,9 A4,9,4,8,1,5,3, output: A1,1 A1,2 A1,4 ... A2,7 A2,8 ... A3,1 A3,2 ... A4,1 A4,3 (5 Replies)
Discussion started by: sdf
5 Replies
Login or Register to Ask a Question