Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

remove first few characters from each line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting remove first few characters from each line
# 1  
Old 05-07-2012
remove first few characters from each line
Hi,

I have a file with lines like below.
I need to remove first few characters from each line until a date format is found.

Code:
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR:
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR:
51129.77605/02/12 11:44:32 NEARRIDGIDAGHLAR@98.251.109.147{E889D36D2B2082668147CB1E0A47A91B.tomcat1}TP-Processor26 LogExchUsage:
18797.45204/13/12 06:55:40 BYE67BYE@96.254.176.116{3FC897AF82897CF3683FE06D6A21997E.tomcat8}TP-Processor18 LogExchUsage: ERROR:
Request for getNextPage.................04/26/12 06:44:20 BAOBEIBUNNY@96.242.252.247{2E29DD4B985715A69EA18021E6EB088B.tomcat4}TP-Processor29 LogExchUsage:

I need to remove first few characters from each line until a date format is found.
out put should be like this.

Code:
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR: 
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR:
05/02/12 11:44:32 NEARRIDGIDAGHLAR@98.251.109.147{E889D36D2B2082668147CB1E0A47A91B.tomcat1}TP-Processor26 LogExchUsage:
04/13/12 06:55:40 BYE67BYE@96.254.176.116{3FC897AF82897CF3683FE06D6A21997E.tomcat8}TP-Processor18 LogExchUsage: ERROR:
04/26/12 06:44:20 BAOBEIBUNNY@96.242.252.247{2E29DD4B985715A69EA18021E6EB088B.tomcat4}TP-Processor29 LogExchUsage:

Any clue...
# 2  
Old 05-07-2012
Code:
$ nawk -F\/ -v OFS=\/ '{if(length($1)>2){$1=substr($1,length($1)-1,2)}}1' input.txt
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR:
05/06/12 20:47:02 GUMGUY@98.192.174.74{42B42A72AC955F5926621273E3A15059.tomcat2}TP-Processor15 LogExchUsage: ERROR:
05/02/12 11:44:32 NEARRIDGIDAGHLAR@98.251.109.147{E889D36D2B2082668147CB1E0A47A91B.tomcat1}TP-Processor26 LogExchUsage:
04/13/12 06:55:40 BYE67BYE@96.254.176.116{3FC897AF82897CF3683FE06D6A21997E.tomcat8}TP-Processor18 LogExchUsage: ERROR:
04/26/12 06:44:20 BAOBEIBUNNY@96.242.252.247{2E29DD4B985715A69EA18021E6EB088B.tomcat4}TP-Processor29 LogExchUsage:

# 3  
Old 05-07-2012
Thanks. It worked, But if a slash '/' (delim) comes in front of the date stamp then the output will be errored?

Code:
99918/797.45204/13/12 06:55:40 BYE67BYE@96.254.176.116{3FC897AF82897CF3683FE06D6A21997E.tomcat8}TP-Processor18 LogExchUsage: ERROR:

will be
Code:
18/797.45204/13/12 06:55:40 BYE67BYE@96.254.176.116{3FC897AF82897CF3683FE06D6A21997E.tomcat8}TP-Processor18 LogExchUsage: ERROR:

# 4  
Old 05-07-2012
ok. make it simpler

Code:
 
nawk -F: -v OFS=":" '{$1=substr($1,length($1)-10,11)}1' input.txt

This User Gave Thanks to itkamaraj For This Post:
ratheeshjulk
# 5  
Old 05-07-2012
Thanks Raj.. But even in the above code there is a chance of error. Because we can expect any characters as unwanted stream of texts in front. I need to start the line with datestamp
Want to know how to find some date time format.
something like
Code:
[0-9][0-9]/[0-9][0-9/[0-9][0-9] [0-9][0-9]:[0-9][0-9:[0-9][0-9]

# 6  
Old 05-07-2012
check this.

But the below will match the incorrect date also ( eg. 88/99/99 xx:xx:xx )

check the regex pattern and correct it

Code:
 
nawk '{if(match($0,"[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]")){print substr($0,RSTART,length($0))}}' input.txt

This User Gave Thanks to itkamaraj For This Post:
ratheeshjulk
# 7  
Old 05-07-2012
remove first few characters from each line
Hi Itkamaraj,

Just want to know why we keep [code] ......}1' input.txt [\code] at the end of code.


Code:
nawk -F: -v OFS=":" '{$1=substr($1,length($1)-10,11)}1' input.txt

Also do we not have any way to read string from backside by speicifying index as -.

Thanks
Krsnadasa

---------- Post updated at 07:47 AM ---------- Previous update was at 07:46 AM ----------

Sorry I mean what is the significance of 1.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove first 2 characters and last two characters of each line

here's what im trying to do. i have a file containing lines similar to this: data.txt: 1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU 1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k... (5 Replies)
Discussion started by: SkySmart
5 Replies

2. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

3. Shell Programming and Scripting

I want to remove 1st and last two characters of each line of the file

I want to remove 1st and last two characters of each line of the file Ex: file1 zzfile1ee @xfile2:y qfile3>> @ file4yy and redirect to the file called new Basically file will have any charcter including space, spical character... Please help.... (7 Replies)
Discussion started by: shell1509
7 Replies

4. Shell Programming and Scripting

Remove characters from line

I have a the following line 22.152.25.36 - K##### "GET /DGGKE/GetMail.do;jwebsphere=FADFFFGSFGSFGSDGFSDFGSDFGSDF HTTP/1.1" 200 44948 Need a cut command which should give me the below output 22.152.25.36 - K##### "GET /DGGKE/GetMail.do HTTP/1.1" 200 44948 Note: The value of jwebsphere can... (6 Replies)
Discussion started by: suindar1982
6 Replies

5. UNIX Desktop Questions & Answers

Remove new line characters from a file

I tried using below command tr -cd "" < InputFile.xml > output.txt ============= This removes all the tabs/newline/extra spaces from a file it successfully removed all the extra spaces,tabs and new line characters but then the complete file become one record. I want to retain one new line... (1 Reply)
Discussion started by: saini
1 Replies

6. UNIX for Advanced & Expert Users

Remove new line characters if found between 1 to 10 columns

Hi, I have a file with ';' delimeter which has some new line characters. How can I delete the new line characters if they are found between 1 to 10 fields. Thanks (3 Replies)
Discussion started by: rudoraj
3 Replies

7. Shell Programming and Scripting

How to remove special characters from each line?

Hello, Is there a simpler way to remove special characters (color codes) from each lines in a log file? I use sed like in the example below but I think there should be a more simple way to achieve the same result: $ cat -vet file1 ^, , , , Maybe to convert the file somehow? ... (5 Replies)
Discussion started by: majormark
5 Replies

8. UNIX for Dummies Questions & Answers

remove characters from line

Hello, I have multiple lines in a file, each of which will have data that looks like this: xxxxxyyyyzzzz4abcdXYZXYZXYZ pqrstPQRST2cdPQRSTPQRST lmnopqr6abcdefgRST.3abc I want to be able to remove the number 4 + the following 4 characters (abcd) in the first line. For the second line,... (1 Reply)
Discussion started by: Gussifinknottle
1 Replies

9. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

10. UNIX for Advanced & Expert Users

how to remove line greater then 3000 characters.

I am using awk and it stops when it encounter line greater then 3000 character. Is there any command which will help me remove line greater then 3000 characters. (10 Replies)
Discussion started by: naren_14
10 Replies
Login or Register to Ask a Question