Separate output based on dates


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Separate output based on dates
# 1  
Old 09-25-2014
Separate output based on dates

Hi guys!

First time poster on here, was wondering if someone could help with a problem which I'm facing.

Is it possible to use awk or sed to separate the below output based on TRANSACTION_DATE? The output would then feed into DataStage.

Contents of CSV file:
Code:
TRANSACTION_DATE_TIME,TRANSACTION_DATE,TRANSACTION_ID,STATUS
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE    
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE    
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE    
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE    
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE    
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE    
6/8/2014 8:25,7/8/2014,MTS_104,COMPLETE


So
Result set 1:
Code:
TRANSACTION_DATE_TIME,TRANSACTION_DATE,TRANSACTION_ID,STATUS
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE    
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE

Result set 2:
Code:
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE    
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE    
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE

Result set 3:
Code:
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE

etc...

Thank you kindly.
Moderator's Comments:
Mod Comment Please use code tags

Last edited by jim mcnamara; 09-25-2014 at 10:51 AM..
# 2  
Old 09-25-2014
Do you need separate output files?- I do not know what Datastage uses.
# 3  
Old 09-25-2014
Hi jim mcnamara,

No, just to be outputted into the unix command line.
# 4  
Old 09-25-2014
Quote:
Originally Posted by Jimmy_the_tulip
Hi guys!

First time poster on here, was wondering if someone could help with a problem which I'm facing.

Is it possible to use awk or sed to separate the below output based on TRANSACTION_DATE? The output would then feed into DataStage.

Contents of CSV file:
Code:
TRANSACTION_DATE_TIME,TRANSACTION_DATE,TRANSACTION_ID,STATUS
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE    
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE    
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE    
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE    
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE    
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE    
6/8/2014 8:25,7/8/2014,MTS_104,COMPLETE


So
Result set 1:
Code:
TRANSACTION_DATE_TIME,TRANSACTION_DATE,TRANSACTION_ID,STATUS
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE    
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE

Result set 2:
Code:
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE    
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE    
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE

Result set 3:
Code:
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE

etc...

Thank you kindly.
Moderator's Comments:
Mod Comment Please use code tags
Hello,

Could you please try following, following will help you to create files according to first column.

Code:
awk -F" |," 'BEGIN{i=1;}{if(A != $1){i++}} {print $0 >> i".txt";A=$1}' Input_file

In this case 3 files will be created as only 3 unique dates are there in given input.

Code:
cat 4.txt
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE
6/8/2014 8:25,7/8/2014,MTS_104,COMPLETE
 
cat 3.txt
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE
 
cat 2.txt
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE

EDIT: Just saw OP wants to show on command line only so following may help.
Code:
sort -k1 Input_file | awk -F" |," 'BEGIN{i=1;}{if(A != $1 && A){print "**************************************"}} {print $0;A=$1}'

Output will be as follows.
Code:
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE
**************************************
5/8/2014 5:10,5/8/2014,MTS_100,COMPLETE
5/8/2014 5:26,5/8/2014,MTS_101,COMPLETE
5/8/2014 5:30,5/8/2014,MTS_102,COMPLETE
**************************************
6/8/2014 8:05,6/8/2014,MTS_103,COMPLETE
6/8/2014 8:25,7/8/2014,MTS_104,COMPLETE

You can print new line etc in place of **** too. Hope this helps.

EDIT: Also if you need to take date as variable then following may help.
Code:
awk -F" |," -vs1="4/8/2014" '{if($1 ~ s1){print $0}}' Input_file

Output will be as follows.
Code:
4/8/2014 2:30,4/8/2014,MTS_99,COMPLETE
4/8/2014 1:50,4/8/2014,MTS_68,COMPLETE

Thanks,
R. Singh

Last edited by RavinderSingh13; 09-25-2014 at 11:34 AM.. Reason: Added a new solution+Added one more solution
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 09-25-2014
Try :

Code:
 awk -F'[ ,]' 'FNR==1{h=$0;next}{f=sprintf("%s.txt",$1); gsub(/\//,"_",f); if(f in F){ print >> f } else { print h RS $0 > f; F[f] } close(f) }' file

---------- Post updated at 09:09 PM ---------- Previous update was at 09:07 PM ----------

If you want to try this on a Solaris/SunOS system, change awk at the start of the script to /usr/xpg4/bin/awk or /usr/xpg6/bin/awk or nawk .
This User Gave Thanks to Akshay Hegde For This Post:
# 6  
Old 09-25-2014
Hi RavinderSingh13,

This was exactly what I was after, thanks a lot!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Call script parameter based on dates

Hi Guys, I am having a script which needs to be iterated based on date passed and itrate based on no of months given. #!/bin/bash var_d=$1 months=$2 sh invoke_script var_d i need to iterate the inside script something like below sh invoke_script 170101 sh invoke_script... (4 Replies)
Discussion started by: Master_Mind
4 Replies

2. Shell Programming and Scripting

To run the script based on dates

I am having below script which needs to be executed based on start and end date #!/bin/bash array=('2016-09-27' '2016-10-27' '2016-11-27' '2016-12-27' '2017-01-27' '2017-02-27' '2017-03-27' '2017-04-27' '2017-05-27' '2017-06-27' '2017-07-27' '2017-08-27' '2017-09-27' ) for i in "${array}" do... (9 Replies)
Discussion started by: rohit_shinez
9 Replies

3. Programming

C++ separate code based on the few changes

Hi, I am working in Visual studio 2008 in which i have written the code in c++,qml,qt.Its a simulator application. I would like to create a simulator with certain changes. In order to do that i have modified the code with few changes based on the preprocessor condition. #define... (4 Replies)
Discussion started by: SA_Palani
4 Replies

4. Shell Programming and Scripting

awk - printing new lines based of 2 dates

I have some test data that is seperated out into annual records, each record has a start date (COL7), an end date (COL8) and a maturity date (COL18) - What I need to do is ensure that there is one record to cover each year right up until Maturity date (COL18). In the first group of the below... (10 Replies)
Discussion started by: Ads89
10 Replies

5. Shell Programming and Scripting

How to separate based on delimiter?

Hi, Variable=MKT1,MKT2,MKT3 and so on i am trying to seperate MKT1,MKT2,MKT3 and store each in a variable. the values in variable1 may vary. I am using bash (8 Replies)
Discussion started by: arghadeep adity
8 Replies

6. Shell Programming and Scripting

How to separate a statement based on some delimiter and store each field in a variable?

Hi, Variable1 = MKT1,MKT2,MKT3,MKT4 Now i want to store each of these value seperated by comma to a array and access each of the values. Also find out number of such values seperated by comma. Variable1 can have any number of values seperated by comma. Thanks :) (3 Replies)
Discussion started by: arghadeep adity
3 Replies

7. Shell Programming and Scripting

Separate string based on delimiter

Hi, for fd in $(grep "/tmp/" hello.properties)The grep gives me the below output: deploydir=/tmp/app1/dfol prodir= /tmp/hello/prop ...... Now i want to store /tmp/app1/dfol then /tmp/hello/prop in a variable so that i can check if those folders files exists or not. The delimiter would... (4 Replies)
Discussion started by: mohtashims
4 Replies

8. Shell Programming and Scripting

script to Find the files based on dates it came

I need a script to get the files for DEC the file format will be B20_abc_ancdfefg_20101209_INCR_201012100145.TXT.gz Based on the bold ones (date) i need to get the files can any one help me plzzzzzz (2 Replies)
Discussion started by: krux_rap
2 Replies

9. Shell Programming and Scripting

To get previous or future dates based on input value

Hi, I need something like, if the input date is 24/Aug/2008 and the inputvalue is +8 then the result should be 1/Sep/2008 (8 days after the input date) if the input date is 24/Aug/2008 and the inputvalue is -8 then the result should be 16/Aug/2008 (8 days before the input date) is there any... (5 Replies)
Discussion started by: Sharmila_P
5 Replies

10. Shell Programming and Scripting

Separate based on file names

Hello experts, This might prove to be a stupid question to some of you, but I have tried to tackle it in different ways. Being new to shell scripting, I am requesting your help in coming up with an elegant solution. I am using Korn shell. We have a directory with file names with the pattern:... (2 Replies)
Discussion started by: prashk15
2 Replies
Login or Register to Ask a Question