Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Question related to sed or awk or perl

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Question related to sed or awk or perl
# 1  
Old 09-30-2009
Question related to sed or awk or perl
Hi All,

I have a big file of 100,000 lines with the following format:

Code:
1,736870,736870,1,2,5,547,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,881246,881246,1,2,6,402,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,975311,975311,1,2,6,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1003112,1003112,1,2,2,93,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1013824,1013824,1,2,5,547,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1014543,1014543,1,2,3,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1016002,1016002,1,2,6,878,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1023369,1023369,1,2,1,1,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445
1,1026987,1026987,1,2,6,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,-g 1253889445

I would like to get rid of the last field in all the lines. Basically I need a file without the last field "-g 1253889445" in every line.

The output file should look like this:
Code:
1,736870,736870,1,2,5,547,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,881246,881246,1,2,6,402,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,975311,975311,1,2,6,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,1003112,1003112,1,2,2,93,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,1013824,1013824,1,2,5,547,1253889535,1253889775,240,30,152.163.141.8,US,0,00, 
1,1014543,1014543,1,2,3,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,1016002,1016002,1,2,6,878,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,1023369,1023369,1,2,1,1,1253889535,1253889775,240,30,152.163.141.8,US,0,00,
1,1026987,1026987,1,2,6,72,1253889535,1253889775,240,30,152.163.141.8,US,0,00,

I am not quite comfortable with sed, awk or perl. Your help is really appreciated.

Thanks,
Toms
Last edited by radoulov; 09-30-2009 at 07:14 AM.. Reason: please use code tags
# 2  
Old 09-30-2009
Backup your data first:


Code:
sed 's [^,]*$  ' infile > _tmp_ && mv -- _tmp_ infile

Some sed implementations support in-place editing with the -i switch,
in that case you don't need to redirection and the mv command.

---------- Post updated at 12:15 PM ---------- Previous update was at 12:13 PM ----------

To keep the forums high quality for all users, please take the time to format your posts correctly.

First of all, use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.)

Second, avoid adding color or different fonts and font size to your posts. Selective use of color to highlight a single word or phrase can be useful at times, but using color, in general, makes the forums harder to read, especially bright colors like red.

Third, be careful when you cut-and-paste, edit any odd characters and make sure all links are working property.

Thank You.

The UNIX and Linux Forums
# 3  
Old 09-30-2009
Code:
$sed -i.BAK 's/,\-g [0-9]\+//g' inputfile

# 4  
Old 09-30-2009
a very easy way is to set the last field to null

Code:
nawk -F"," '{ $NF="" } 1 ' OFS=","   input_file


Best Regards
# 5  
Old 09-30-2009
Quote:
Originally Posted by ahmad.diab
a very easy way is to set the last field to null

Code:
nawk -F"," '{ $NF="" } 1 ' OFS=","   input_file

Best Regards
Or if you have GNU awk:

Code:
gawk -F, NF-- OFS=, infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk related question

awk -F ";" 'FNR==NR{a=$1;next} ($2 in a)' server.list datafile | while read line do echo ${line} done when i run the above, i get this: 1 SERVICE NOTIFICATION: nagiosadmin skysmart-01.sky.net .... instead of: SERVICE NOTIFICATION: nagiosadmin skysmart-01.sky.net .... can... (4 Replies)
Discussion started by: SkySmart
4 Replies

2. Shell Programming and Scripting

Perl sed question

Hi, I'm trying to use perl the way I use sed. Lets say I have this command: $ echo abc | sed 's/abc/&&/' abcabcIf I try this with perl, I end up with this: $ echo abc | perl -pe 's/abc/&&/' &&How do I make the '&' sign work? Another thing that I can't get with perl is this: $ echo... (4 Replies)
Discussion started by: Subbeh
4 Replies

3. Shell Programming and Scripting

awk related question (for loop) difficult scenario

VARA='hello|welcome|gone|fantastic|superb|nicecar' if VARA contains a pipe "|", i want the contents of VARA to be tranformed to: VARA="(hello) (welcome) (gone) (fantastic) (superb) (nicecar)" so that, when i echo the contents of $VARA like this: echo "$VARA" or like this: print... (8 Replies)
Discussion started by: SkySmart
8 Replies

4. Shell Programming and Scripting

Perl related question

hi, iam perl begginer,i have written the program #!/usr/bin/perl #use warnings; use strict; print "Enter the name:","\n"; my $name=<STDIN>; my %hash=(siva => "9902774481", dev => "9916391244", venky => "9440506760", manohar => "9440232695" ); print "$name no is:... (5 Replies)
Discussion started by: siva.hardwork
5 Replies

5. Shell Programming and Scripting

awk related question

awk "/^<Mar 31, 2012 : /,0" /app/blah.log can someone please help me figure out why the above command isn't pulling anything out from the log? basically, i want it to pull out all records, from the very first line that starts with the date "Mar 31, 2012" and that also has a time immediately... (4 Replies)
Discussion started by: SkySmart
4 Replies

6. Shell Programming and Scripting

sed or awk question

Hello expert, I have an output file with few thousand lines similar like below : &quot;Future Netmgmt&quot; &quot;10.99.16.0&quot; &quot;N&quot; &quot;10&quot; &quot;10.0.0.0&quot; &quot;Circuitless-IP&quot; &quot; &quot; &quot;255.255.254.0&quot; &quot;Future Netmgmt&quot; &quot;10.99.18.0&quot; &quot;N&quot; &quot;10&quot; &quot;10.0.0.0&quot; &quot;Circuitless-IP&quot; &quot; &quot; &quot;255.255.254.0&quot; &quot;WAAS loopbacks&quot; &quot;10.99.20.0&quot;... (6 Replies)
Discussion started by: dannytrinh
6 Replies

7. UNIX for Dummies Questions & Answers

sed and awk question

Hi, I have to write a script that goes through every *.cpp file in the current directory and if any file has #includes of non-system header files (those with double quotes around them), then I need to print out those header files within the quotes. I've figured out how to run a for loop and find... (4 Replies)
Discussion started by: MEllis5
4 Replies

8. Shell Programming and Scripting

to sed or awk, that is the question?

I have a text file with about 790 lines, at the end of many of the lines there is the text string 'f4' I want this text to be 'f2' What is the best way to do this? is it sed or awk? or something else? how? Also, there is maybe fifty occasions where f4 appears near the begining of the line that I... (2 Replies)
Discussion started by: ajp7701
2 Replies

9. Shell Programming and Scripting

sed and awk question

hello, I have this in a file server_name=DB1 hostname=db1 I want to change hostname value to `hostname`. Any idea? and server_name value to toUPPER (`hostname`). Any idea? thanks (3 Replies)
Discussion started by: melanie_pfefer
3 Replies

10. UNIX for Advanced & Expert Users

A question on using sed or awk

Hi I have a pattern like this. repeating many lines CHANGE #13 TYP:22 CLS: 21 AFN:12 DBA:0x0040a15f OBJ:41142 SCN:0x0000.00036b3e SEQ:1 OP:11.2 CHANGE #15 TYP:32 CLS: 32 AFN:212 DBA:0x0040a15f OBJ:41143 SCN:0x0000.00046b3e SEQ:1 OP:13.3 . And i am trying to do the following: a) I need to get... (2 Replies)
Discussion started by: hare
2 Replies
Login or Register to Ask a Question