Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File modify

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File modify
# 1  
Old 05-14-2015
File modify
Hi All

I am getting a file with below pattern -

Code:
00150366 05/08/2015 07:14:32
8000186167+++ 50195281000000000371001010903236
800186167+++ 100209000000000
800000018617+++ 50295281000000000371001010900217================================3u4398482344334=432434
00150367 05/08/2015 07:14:32
80009000001+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
8000900000186167+++ 50295281000000000371001010900217=================
00150366 05/08/2015 07:28:32
8000900186167+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
900000186167+++ 50295281000000000371001010900217=================


I have to sort a file based on the first column for ex - 00150366 in the bold pattern in all the file - remove if there is anything duplicate ... and keep the latest time and date format one .

and then remove the bold three columns and process the file as these columns are just for sorting - Every Bold Pattern will have 3 records inside .

If every record had these 3 columns in front of it then it is simple . But i am not understanding with all these spaces now .

Please let me know in case anyone knows if this is possible or not .
Last edited by rbatte1; 05-14-2015 at 11:23 AM.. Reason: Added CODE tags and removed unnecessary font/colour settings
# 2  
Old 05-14-2015
Not sure I understand that spec. Try
Code:
paste -sd"\t\t\t\n" file | sort -k3,3 | tail -1 | tr '\t' '\n'
00150366 05/08/2015 07:28:32
8000900186167+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
900000186167+++ 50295281000000000371001010900217=================

This would fail if e.g. the time range would span two or more days, if other than four lines per records were present, and maybe other reasons.
This User Gave Thanks to RudiC For This Post:
honey26
# 3  
Old 05-14-2015
Hi Rudic

Thanks for your help

May be this will help
The below should be the output -

Code:
80009000001+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
8000900000186167+++ 50295281000000000371001010900217=================
8000900186167+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
900000186167+++ 50295281000000000371001010900217=================


basically sort out on the unique number 00150366 and remove if there are any duplicates for this unique and choose the latest one . and send all the unique ones . Removing these fields unique number and date time ....
Last edited by Don Cragun; 05-15-2015 at 04:11 AM.. Reason: Remove BOLD tags; add CODE and ICODE tags.
# 4  
Old 05-14-2015
Please use code tags as required by forum rules!

Try
Code:
paste -sd"\t\t\t\n" file | sort -k1,1 -k3,3r  | awk '!T[$1]++ {sub ($1 " " $2 " " $3 "\t", ""); gsub ("\t", "\n"); print}'
8000900186167+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
900000186167+++ 50295281000000000371001010900217=================
80009000001+++ 50195281000000000371001010903236
8000900000186167+++ 50695281000000000371001010953001Y
8000900000186167+++ 50295281000000000371001010900217=================

This User Gave Thanks to RudiC For This Post:
honey26
# 5  
Old 05-17-2015
Awesome it worked perfectly .. Smilie Smilie ....
Thanks a lot buddy Smilie

I will analyze how this is working . thanks again Smilie
# 6  
Old 05-26-2015
Hi Rudic

The Time thing is working But the date thing is not working .
If the dates are different it is not picking up with the latest date .

Instead picking up with time for older date . Can you please help .
# 7  
Old 05-26-2015
RudiC already told you that that script would only work if all of the times for a given field 1 were on the same date. We would hope that you would have used his suggestion as a starting point and would have figured out how to extend it to work with multiple dates.

Since your sample data was all on one day, testing for anything else is just guess work. (I'm guessing that the sample data you provided is for May 8, 2015; not August 5, 2015. If I guessed incorrectly, you'll have to modify the sort to make the month a higher order sort key than the day.)

The following produces the correct sets of lines for you, but the order of the sets of lines is different than in your sample. You didn't specify any order for the output other than that the records selected of output had to be the most recent date and time for each key value. If you want a specific order for the keys, you should have said what that order should be. Try:
Code:
paste -sd"\t\t\t\n" file | sort -k2.7,2r -k2.1,2.5r -k3,3r  | \
    awk '!T[$1]++ {sub("[^\t]*\t", ""); gsub("\t", "\n"); print}

This User Gave Thanks to Don Cragun For This Post:
honey26
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ed to modify a file --- or not?

Running Oracle Linux 6 (derivative of RHEL 6) Given this snippet of code in a shell script: #-- reset oratab to use 11.2 home for dwdev #-- normally we'd just use sed to do this sort of thing, but that would #-- require permissions that we don't have in the /etc/ directory, so we #-- ... (3 Replies)
Discussion started by: edstevens
3 Replies

2. Shell Programming and Scripting

Modify CSV file

Hi, I would like to change my CSV file by adding " and : and moving some of the information around. the CSV file looks as follows: 501254424;500440257;PE PACKS;300467279;PREP;;276476070;655031001867176;Two Block;Olga;25/12/2015 00:00:00;Olga I would like to move the field 7 to the front "... (13 Replies)
Discussion started by: omuhans123
13 Replies

3. Shell Programming and Scripting

Modify a file by another file: add new line and variable after string is found

hello, I have problem with writing/adjusting a shell script. I searched forum and unfortunately couldn't write scipt based on the information I found. I never wtire such so it's hard for me and I do need to modify one script immediately. case looks like: 1. 'file' that needs to be modified... (3 Replies)
Discussion started by: bipbip
3 Replies

4. Shell Programming and Scripting

ksh- redirect stderr to file and then modify the file

I have the following: remsh $host -n 2>>syslog_issue_list.txt grep -i -e "EMS" -e "error" -e "warning" -e "excessive" /var/adm/syslog/syslog.log | awk /"$DATE1"/ | awk -vhost="$host" '!/remsh|telnetd/{print host "\n", $0 >> "syslog_issue_list.txt"}' I am creating a health script that has... (4 Replies)
Discussion started by: chipblah84
4 Replies

5. Shell Programming and Scripting

Modify file

Hi, I have a file that looks like this: 27+:<10,289808,1> 31+:<11,1445372,1> 33-:<7,1014101,2> 35+:<11,728811,1> 36-:<11,1445205,0> 37+:<11,1445792,2> and I want to change it to this: + 10 289808 + 11 1445372 - 7 1014101 + 11 728811 - 11 1445205 + 11 1445792 (3 Replies)
Discussion started by: kylle345
3 Replies

6. Shell Programming and Scripting

Perl : how to modify a file without generate a temporary file

Hi All, I have a file like below, how can i insert one line after line 1 without using a temporary file in perl? line 1 line 2 line 3 expected result line 1 new line <---insert here line 2 line 3 (2 Replies)
Discussion started by: summer_cherry
2 Replies

7. UNIX for Dummies Questions & Answers

Modify Text File

Hi, I would like to remove any lines from a text file that begin with #, or that are blank. How can I do that with BASH? Mike (3 Replies)
Discussion started by: msb65
3 Replies

8. Shell Programming and Scripting

Modify a file

Hi all Can anyone suggest me a good solution ? My requirement is as follows I have a plain text file similar to this... sending data to 0003345234 here is the output... ,.......... ........... ....... sending data to 00033452ab here is the output... ,.......... ........... .... (5 Replies)
Discussion started by: ./hari.sh
5 Replies

9. Shell Programming and Scripting

Can I modify the .bashrc file instead of .profile file to customize my login?

Hello, I got this question which tells me to customize my login script. Some people in the forums suggested to modify the .profile file in my home directory. I did so, but none of my customizations show up when I open the terminal after. So, I tried to modify other files in my home directory,... (1 Reply)
Discussion started by: Hyunkel
1 Replies

10. Shell Programming and Scripting

modify a file

I'm a new member of the forum, I had this new generate file since I use 'grep' and 'awk', what I want to do is get rid off the all 0s before the numbers, is there any one who could help me to figure it out? Thanks a lot! yun 0000000029 000q7472 2002/03/01 0000000030 000q7472 2002/03/01 ... (2 Replies)
Discussion started by: yxiao
2 Replies
Login or Register to Ask a Question