Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed --> sort data by date

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed --> sort data by date
# 1  
Old 01-02-2014
sed --> sort data by date
Hi,

i "tried" to sort data by date. So far, i used sed to take the data from the last and the actual month. Now, after changing the year it is not working properly.
Code:
i use:
GNU bash, version 4.2.45(1)-release (x86_64-suse-linux-gnu)

sed -n '/[0-3][0-9]\/['$y']/p' $Home/../scripte/pd_0.txt

y is a variable, build bei the shell-command y=`date +%y`

### data-format in pd_0.txt
2985,12/31/13,23:30,92637,9135,10018,9758,8504,6555,4573,2834,...
2986,12/31/13,23:40,97195,9281,10080,9868,8752,7091,4792,2951,...
2987,12/31/13,23:50,95258,9453,10164,10154,9285,7146,4919,2982...
2988,01/01/14,00:00,100358,9745,10622,10376,9291,7475,5222,3395,...
2989,01/01/14,00:10,128834,11704,13456,14235,13857,12485,10119,...
2990,01/01/14,00:20,165685,13613,16164,17818,18713,18242,15764,....

after all i'am very confused about. espacially why it works properly last year.
what i want to filter, ist the data from the last two month - e.g. from today, the data from Dec 2013 and the Data from Jan 2014.
Can someone help me please.

Thanks in advance,
IMPe
# 2  
Old 01-02-2014
A best practice is to use the YYYYMMDD format (date '+%Y%m%d') for the date, so that further sorting are made easy.

Not that YYYY is better than YY because retaining only the 2 last digit suppose a loss of information and may lead to ambiguous situations.
Last edited by ctsgnb; 01-02-2014 at 07:18 AM..
This User Gave Thanks to ctsgnb For This Post:
IMPe
# 3  
Old 01-02-2014
Quote:
Originally Posted by ctsgnb
A best practice is to use the YYYYMMDD format (date '+%Y%m%d') for the date, so that further sorting are made easy.

Not that YYYY is better than YY because retaining only the 2 last digit suppose a loss of information and may lead to ambiguous situations.
Hi ctsgnb,

thanks for your fast repley.
what i forgot to say: the meassurement data and data-format is build by a device. it is not possible for me to change it. So i choose it this way, because the data-format is fix and csv: MM/DD/YY, hh:mm, 1234, 5678 .

Thanks in advance,
IMPe
# 4  
Old 01-02-2014
To keep the same formatting:
Code:
sed 's;[:/];,;g' yourfile | sort -t, -k 4,4 -k 2,3 -k 5,6 | sed 's/,/:/5;s:,:/:3;s:,:/:2'

or move the YY at the beginning of the date so that the format will change into YY/MM/DD:
Code:
sed 's:\(../..\)/\(..\):\2/\1:' yourfile | sort -t, -k 2,3

As mentionned in my previous post, retaining only YY loose information and may lead to unexpected result (example: 12 may mean 1912 or 2012 or whatever ???12...)

---------- Post updated at 14:43 ---------- Previous update was at 13:07 ----------

I just got aware i've missunderstood your question:

Before running your
Code:
sed -n '/[0-3][0-9]\/['$y']/p' $Home/../scripte/pd_0.txt

You need to check wether you are in january : if so, the %y should be 134 rather than 14
Code:
if [ $(date +%m) = "01" ]
then
y=$(echo $(date +%y)$( echo $(($(date +%Y)-1 )) | tail -c 3 ) )
else
y=$(date +%y)
fi

should be sufficient since [1413] in regex will give the same result than [134] (it's just a little less clean)

By the way, you will have to care of century jump because your %y will go from 99 to 00, that's why i calculate last year using %Y (uppercase) rather than %y.
Last edited by ctsgnb; 01-02-2014 at 08:28 AM..
This User Gave Thanks to ctsgnb For This Post:
IMPe
# 5  
Old 01-03-2014
Quote:
Originally Posted by ctsgnb
.....
Code:
if [ $(date +%m) = "01" ]
then
y=$(echo $(date +%y)$( echo $(($(date +%Y)-1 )) | tail -c 3 ) )
else
y=$(date +%y)
fi

should be sufficient since [1413] in regex will give the same result than [134] (it's just a little less clean)

By the way, you will have to care of century jump because your %y will go from 99 to 00, that's why i calculate last year using %Y (uppercase) rather than %y.
Hi ctsgnb,

thanks a lot for your extensive support. It take some time for me, to understand all your methods and proposals. But again i have learned more about sed and i solved my problem properly.

Thanks a lot,
IMPe
# 6  
Old 01-03-2014
Perl Command Usage
Hi All,

Actually I am new to this Forum.Don't know how to post my question.

Can anyone help me :

we can replace dbSQL( with dbSQLWithBind( value by using :
:.,$s/dbSQL(/dbSQLWithBind(/g command inside a script.

For replacing ^M characters from a file, we normally use this command:

perl -i -p -e 's/^M//g' *.txt

But i am failing in replacing dbSQL( with dbSQLWithBind( value through perl command.
Please help me in replacing the value dbSQL( with dbSQLWithBind( globally in all scripts without opening the scripts.
Last edited by Vara Prasad; 01-03-2014 at 06:08 AM.. Reason: Added comments more
# 7  
Old 01-03-2014
for Vara Prasad:

Code:
cat file | sed 's/\^M//g'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sort by name and date

nawk '$1=="Date" {d=$(NF-2);next} $1=="Queue" {q=$NF;next} $1=="Forms"{print q, $NF, d}' OFS='|' printfile.log I have this script working. Please let me know how to sort by Queue and then Date. (4 Replies)
Discussion started by: Daniel Gate
4 Replies

2. Shell Programming and Scripting

Sort, sed, and zero padding date column csv bash scripting

Hello people, I am having problem to sort, sed and zero padding of column in csv file. 7th column only. Input of csv file: 1,2,3,4,5,6,4/1/2010 12:00 AM,8 1,2,3,4,5,6,3/11/2010 9:39 AM,8 1,2,3,4,5,6,5/12/2011 3:43 PM,8 1,2,3,4,5,6,12/20/2009 7:23 PM,8 Output:... (5 Replies)
Discussion started by: sean1357
5 Replies

3. Shell Programming and Scripting

Sort data by date first and then remove duplicates

Hi , I have below data inside a file named ref.psv . I want to create a shell script which will do the below 2 points : (1) sort the file content first based on the latest date which is the last column in the file (actual file its the 175th column) (2)after sorting the file based on latest date... (3 Replies)
Discussion started by: samrat dutta
3 Replies

4. Shell Programming and Scripting

Sort data by date and then search by column

Hi, I have a file where data is pipe separated.First i want to sort the file content by date . Then i want to pick up the records based on the first column which should be unique and not have duplicates. NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15... (2 Replies)
Discussion started by: samrat dutta
2 Replies

5. Shell Programming and Scripting

Sort help: How to sort collected 'file list' by date stamp :

Hi Experts, I have a filelist collected from another server , now want to sort the output using date/time stamp filed. - Filed 6, 7,8 are showing the date/time/stamp. Here is the input: #---------------------------------------------------------------------- -rw------- 1 root ... (3 Replies)
Discussion started by: rveri
3 Replies

6. Shell Programming and Scripting

sort the date

Hi All, Please help me to sort the date field which is in the format 2012-02-03 16:09:37.388... Platform: Red Hat linux Thanks in advance (2 Replies)
Discussion started by: jesu
2 Replies

7. Programming

sort data by date.. pls help

hi all, could anyone help me? I need to query output by compare dates from 2 table and i'm using a UNION query..and wanted to sort the output by date.. My query like this: SELECT TO_CHAR(DATE) DATE1, INVOICE FROM ACCOUNT1 WHERE DATE < (to_date('122003','MMYYYY')) UNION ... (2 Replies)
Discussion started by: kate katherine
2 Replies

8. UNIX for Dummies Questions & Answers

how to combine du -h and sort by date

The problem: I have a task to give a my boss: 1. the files and directory sizes in human readable form: du -h 2. He also wants me to give him the date stamp on the files and directories. something like ls -R or sort by date Does anyone have a script that can do this efficiently (I am... (2 Replies)
Discussion started by: obology
2 Replies

9. Shell Programming and Scripting

how to sort by the date

Hello World~ Please Help Me(BASH) input: dde,2007.8.25,891 dde,2007.8.23,356 dfe,2007.10.12,341 cba,2005.12.5,342 I wanna know how to sort by the date(2005.12.5) output: cba,2005.12.5,342 dde,2007.8.23,356 dde,2007.8.25,891 dfe,2007.10.12,341 Thanks in advance (3 Replies)
Discussion started by: lifegeek
3 Replies

10. Shell Programming and Scripting

Sort by Date

I'm looking to edit a file which contains various data including date.(ddmmyyyy) I want to sort by date and then count the number of different dates found Any ideas how to acheive this Thanks in advance. (2 Replies)
Discussion started by: Mudshark
2 Replies
Login or Register to Ask a Question