Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep only specific lines ordered by column/date

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep only specific lines ordered by column/date
# 1  
Old 09-25-2011
Grep only specific lines ordered by column/date
Hi everybody,
I'd like to think I've been through the search tool not only on this site, but also on google too, but I haven't been able to find what I was looking for. If I might've missed something on this forum, please slap me in the face with a link that you consider useful for my query Smilie

I got a bunch of backup jobs that I need to order and grep only for those with errors:
Code:
926717 Image De   Done      0                                                                                                                       08/28/11 07:58:30  08/28/11 08:01:08  000:02:38              1
926709 Image De   Done      0                                                                                                                       08/28/11 05:40:05  08/28/11 05:43:03  000:02:58              1
926679 Image De   Done      0                                                                                                                       08/28/11 03:51:07  08/28/11 03:53:15  000:02:08              1
938526   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               09/13/11 20:00:00  09/13/11 22:00:03  002:00:03              1
933442   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               09/06/11 20:00:00  09/06/11 22:00:03  002:00:03              1
928458   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               08/30/11 20:00:00  08/30/11 22:00:03  002:00:03              1
945927   Backup   Done    196       SQL_SERVER_FALCON_BACKUP                       Full                           BAZ                               09/23/11 21:00:00  09/23/11 23:00:03  002:00:03

Notice there are jobs which finish with status code 0 (on column 4 - meaning they're successful) and those with 196 (un-successful). I would like to print out only the lines containing jobs that DON'T finish successfully and order them by date. What I've managed to do was to use the sort -k 4,7 (column 7 is the starting date of the job) but I would also need to exclude from the printed lines those that don't matter to me (for example jobs that have failed 2 days ago... if we're on Sept 25, I'd like to have only lines containing failed jobs from Sept 25, NOT Sept 23rd). I think this is easy for most of the gurus here, but I'm just a storage guy, so do forgive my lack of knowledge on this Smilie
Last edited by Scott; 09-25-2011 at 08:32 AM.. Reason: Please use code tags to preserve formatting...
# 2  
Old 09-25-2011
It would be simpler if the date was stored in YYYYMMDD format.

Some of your field numbers were wrong. Field "2", for example, is actually two fields in some cases. It's possible that the actual output is tab-delimited, but you just didn't paste it properly here?

This will add an extra field with that data format, then sorting is easier...

Code:
$ awk '$2=="Backup" && $4 {split( $8, D, "/"); T=$9; gsub( /:/, "", T); print D[3]D[1]D[2]T, $0}' file1 | sort -nk1
110830200000 928458   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               08/30/11 20:00:00  08/30/11 22:00:03  002:00:03              1
110906200000 933442   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               09/06/11 20:00:00  09/06/11 22:00:03  002:00:03              1
110913200000 938526   Backup   Done    196      ADDICTION_TRADEFIN_EXPORT                       Full                     addiction                               09/13/11 20:00:00  09/13/11 22:00:03  002:00:03              1
110923210000 945927   Backup   Done    196       SQL_SERVER_FALCON_BACKUP                       Full                           BAZ                               09/23/11 21:00:00  09/23/11 23:00:03  002:00:03

The last bit about getting the last two days I leave to you (or anyone else) Smilie.
This User Gave Thanks to Scott For This Post:
dilibau
# 3  
Old 09-25-2011
I think I got what I wanted by doing just a little bit more search Smilie

bpdbjobs | sort -rk 4,7 | awk '{ if ($4 == "196") print $0 }' | sort -rk 8

That gives me something like this:

Code:
946571   Backup   Done    196                          thumb           Differential-Inc                         thumb                               09/24/11 18:35:00  09/25/11 00:00:08  005:25:08              1
946570   Backup   Done    196                          thumb           Differential-Inc                         thumb                               09/24/11 18:35:00  09/25/11 00:00:08  005:25:08              1
946569   Backup   Done    196                          thumb           Differential-Inc                         thumb                               09/24/11 18:35:00  09/25/11 00:00:08  005:25:08              1
946278   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946277   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946276   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946275   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946274   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946273   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946272   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946271   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
946270   Backup   Done    196                     dollar_new                       Full                        Dollar                               09/24/11 04:01:27  09/24/11 06:00:03  001:58:36              1
945954   Backup   Done    196                 atm_monitoring                       Full atm_monitoring.alkhaliji.bank                               09/23/11 21:25:00  09/24/11 06:00:03  008:35:03              1
945953   Backup   Done    196                 atm_monitoring                       Full atm_monitoring.alkhaliji.bank                               09/23/11 21:25:00  09/24/11 06:00:03  008:35:03              1
945952   Backup   Done    196                 atm_monitoring                       Full atm_monitoring.alkhaliji.bank                               09/23/11 21:25:00  09/24/11 06:00:03  008:35:03

# 4  
Old 09-25-2011
Oh, yes, I should have been looking for "196", than assuming "not 0" meant error.

The odd American date format might save you this year, but not sure how well that sort will hold up when 2012 comes along.

Which would come first?
Code:
09/23/11
01/01/12

At least with most other date formats, you'd only have to wait until October for it to break - less time to forget what you did!
Code:
23/09/11
01/10/11

But the international format will never break
Code:
20120101
20110923

Did I labour that point too much? Smilie
This User Gave Thanks to Scott For This Post:
dilibau
# 5  
Old 09-25-2011
If predictions are right, it's not only the American date format that will suffer, but all of us... Smilie and yeah, thanks for the effort Scott, but you went way beyond what I actually needed... It's like in string theory: it's absolutely wonderful mathematically, but no one can prove it right...Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with Grep Specific Date From One File Into A New One

Hello All, First post, don't know much about Linux/Unix, but I need some help. Normally, I do grep 'what I'm searching for' FILE > file.txt so I can pull data from one file, and put it into a new one. The issue I am having is with specific dates, since the date field is 4, and it appears... (3 Replies)
Discussion started by: DennisG34
3 Replies

2. Shell Programming and Scripting

Can grep a specific column?

Hello All I have an input file with data below. I would like to grep and display the data where 3rd column contains string or at least one character. Kindly please help me with this! Input: tjfa3|zznpou|224fdd.34.ff3.35 |Tiv|Otj|1 fgduul7|zznikj| ... (7 Replies)
Discussion started by: DoveLu
7 Replies

3. UNIX for Dummies Questions & Answers

Highlighting specific column in cat/grep

I have a large * delimited text file and need to highlight specific columns on specific lines. The file looks similar to this: ABC*01*00*01*00000 000000 *00000 000000 *35*0*0*0*0*20000*0*0*1*0000*00*4*0*3*0*0*1*0* *35*0000*001*4*1OT2*0148*0*0*0*0*0*0*6A7801B0**TEST1... (1 Reply)
Discussion started by: say170
1 Replies

4. Shell Programming and Scripting

Grep on a value in a specific column

I need to grep all records from a file that has 1072 in 3rd column. 1072 can be prefixed by "SBC.", "CLS." or "DPT.". My search is just based on string 1072 in 3rd column. Delimiter in the file is tab. For example: Input FIle: "InvType" "Organization" "SBC.10720101" "CP.BUP.NY" "InvType"... (4 Replies)
Discussion started by: yale_work
4 Replies

5. UNIX for Dummies Questions & Answers

Grep specific lines

Hello I have a file with nearly 90000 lines in x,y,z format but have some lines that I do not need to show. Is there anyway to delete those 3 lines after every 288 lines. Eg I keep the first 288 lines delete (289, 290 291); keep the next 288 lines after those and so on... Thanks (6 Replies)
Discussion started by: Madiouma Ndiaye
6 Replies

6. Shell Programming and Scripting

How to awk or grep the last column in file when date on column contains spaces?

Hi have a large spreadsheet which has 4 columns APM00111803814 server_2 96085 Corp IT Desktop and Apps APM00111803814 server_2 96085 Corp IT Desktop and Apps APM00111803814 server_2 96034 Storage Mgmt Team APM00111803814 server_2 96152 GWP... (6 Replies)
Discussion started by: kieranfoley
6 Replies

7. Shell Programming and Scripting

Extract lines with unique value using a specific column

Hi there, I need a help with extracting data from tab delimited file which look like this #CHROM POS ID REF ALT Human Cow Dog Mouse Lizard chr2 3033 . G C 0/0 0/0 0/0 1/1 0/0 chr3 35040 . G T 0/0 0/0 ./. 1/1 0/1 chr4 60584 . T G 1/1 1/1 0/1 1/1 0/0 chr10 7147815 . G A 0/0 1/1 0/0 0/0... (9 Replies)
Discussion started by: houkto
9 Replies

8. Shell Programming and Scripting

Grep all lines for a specific date in log-files

I need to grep all lines for "yesterday" in /var/log/messages. Dates are in the format "YYYY-MM-DD". (5 Replies)
Discussion started by: Padmanabhan
5 Replies

9. Shell Programming and Scripting

script for get lines with specific date

I'm a newbie in AIX, i want to make a script for grep any lines with date bellow 20 PRINT0089-88615 data1 3072 Mon Dec 19 17:53:49 WITA 2011 PRINT0089-88616 data1 4096 Mon Dec 19 17:53:49 WITA 2011 PRINT0089-88618 data1 5120 Mon Dec 19... (7 Replies)
Discussion started by: michlix
7 Replies

10. Shell Programming and Scripting

sed replace values of column with ordered numbering

Hello, I am trying to write a shell script that will create a gnuplot file. My main problem is that I have a data file with two columns: 1.05929120E+09 5.0000701214792 1.05930096E+09 5.00006386985764 1.05930584E+09 5.00019465404908 1.05931072E+09 5.00031960589719 ... (2 Replies)
Discussion started by: pau
2 Replies
Login or Register to Ask a Question