Using tr, sed or awk to delete text from nth column only


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using tr, sed or awk to delete text from nth column only
# 1  
Old 09-02-2011
Using tr, sed or awk to delete text from nth column only

Hi everyone, this is my first post here, I hope someone can help me.

I have a file which I need to delete characters '_F3' from the end of the text in the first column. The problem is that the characters may also occur elsewhere in the file (i.e. second columns onwards). I tried sed (thinking I was a linux genius) and then realised that there were rows where the character is not in the first column, but appears in a later column, and I was deleting the second occurrence of '_F3' when I only actually want to delete it if it is in the first column.

The command I was using was:
PHP Code:
sed 's/_F3//' filename newfilename 
I need to retain everything in the file except the trailing _F3 in the first column, and write to a new file.

I have spent ~3h trying to find a solution to this, and think its probably an awk command I need, but I am afraid my awk skills are 0. Smilie

Please can someone help me out!

Many thanks
Helen
# 2  
Old 09-02-2011
Welcome the the forum.

This might help:

Code:
awk '{gsub("_F3","",$1)}1' filename > newfilename

# 3  
Old 09-02-2011
Yes that has worked, but unfortunately I seem to have lost the formatting of my file now. I think it was tab delimited before, now it just has a blank space between columns, and my next process wont accept this.

Helen

Before:
PHP Code:
1329_105_1480_F3        355     chr1    13484   1       50M     =       13572   123     CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG      ,,@NPYG423BC553AC.2BPB;:7OH0.-=><1,I3!5=D<4)-OD=44      NM:i:0  NH:i:4  CC:Z:chr12      CP:i:92080      XS:A:+  HI:i:0
1863_1224_411_F3        99      chr1    13484   3       50M     
=       13572   123     CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG      UUZ_[][VXY[XRNOYJFURZZULJZ_ZOQ[SRRTW@CPBJHJWU\FAMM      NM:i:0  NH:i:2  CC:Z:chr2       CP:i:114357483  XS:A:+  HI:i:
After:
PHP Code:
1329_105_1480 355 chr1 13484 1 50M 13572 123 CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG ,,@NPYG423BC553AC.2BPB;:7OH0.-=><1,I3!5=D<4)-OD=44 NM:i:0 NH:i:4 CC:Z:chr12 CP:i:92080 XS:A:+ HI:i:0
1863_1224_411 99 chr1 13484 3 50M 
13572 123 CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG UUZ_[][VXY[XRNOYJFURZZULJZ_ZOQ[SRRTW@CPBJHJWU\FAMM NM:i:0 NH:i:2 CC:Z:chr2 CP:i:114357483 XS:A:+ HI:i:
# 4  
Old 09-02-2011
Try this
Code:
$ awk '{gsub("_F3","",$1)}1' FS=OFS file
1329_105_1480        355     chr1    13484   1       50M     =       13572   123     CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG      ,,@NPYG423BC553AC.2BPB;:7OH0.-=><1,I3!5=D<4)-OD=44      NM:i:0  NH:i:4  CC:Z:chr12      CP:i:92080      XS:A:+  HI:i:0
1863_1224_411        99      chr1    13484   3       50M     =       13572   123     CAGCTGCACCACTGCCTGGCGCTGTGCCCTTCCTTTGCTCTGCCCGCTGG      UUZ_[][VXY[XRNOYJFURZZULJZ_ZOQ[SRRTW@CPBJHJWUFAMM      NM:i:0  NH:i:2  CC:Z:chr2       CP:i:114357483  XS:A:+  HI:i:0  
$

# 5  
Old 09-02-2011
Quote:
Originally Posted by hlwright
... , but unfortunately I seem to have lost the formatting of my file now. I think it was tab delimited before, now it just has a blank space between columns, and my next process wont accept this.
Helen
Code:
awk '{a=x;l=split($1,_,"_");if(_[l]=="F3"){for(i=0;++i<l;){a=(a?a"_":x)_[i]};sub($1,a)}}1' file


Last edited by danmero; 09-02-2011 at 09:43 AM.. Reason: Add fail-safe
# 6  
Old 09-02-2011
Danmero

Thank you very much. The script is taking a long time to run but looks like it is going to work perfectly (works on a small extract of the file).

Thank you so much for your help. Smilie

Helen
# 7  
Old 09-02-2011
Code:
perl -plne 's/^(\w+)_F3/$1/' datafile

This User Gave Thanks to durden_tyler For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to print nth till last column of ls output using sed

I wish to print first, third and sixth till the last column from the output of ls command ls -ltr /app/deploy.yml -rw-rw-r-- 1 user1 dba 27342 Aug 28 10:17 /app/deploy.yml Desired Output: Below command gives me the desired output. ls -ltr /app/deploy.yml | awk '{$2=$4=$5=""; print... (6 Replies)
Discussion started by: mohtashims
6 Replies

2. Shell Programming and Scripting

Append data with substring of nth column fields using awk

Hi guys, I have problem to append new data at the end of each line of the files where it takes whole value of the nth column. My expected result i just want to take a specific value only. This new data is based on substring of 11th, 12th 13th column that has comma seperated value. My code: awk... (4 Replies)
Discussion started by: null7
4 Replies

3. Shell Programming and Scripting

Taking nth column and putting its value in n+1 column using awk

Hello Members, Need your expert opinion how to tackle below. I have an input file that looks like below: USS|AWCC|AFGAW|93|70 USSAA|Roshan TDCA|AFGTD|93|72,79 ALB|Vodafone|ALBVF|355|69 ALGEE|Wataniya (Nedjma)|DZAWT|213|50,550 I like output file in below format: ... (7 Replies)
Discussion started by: umarsatti
7 Replies

4. Shell Programming and Scripting

awk search and replace nth column by using a variable.

I am passing a variable and replace nth value with the variable. I tried using many options in awk command but unable to ignore the special characters in the output and also unable to pass the actual value. Input : "1","2","3" Output : "1","1000","3" TempVal=`echo 1000` Cat... (2 Replies)
Discussion started by: onesuri
2 Replies

5. Shell Programming and Scripting

awk to search for specific line and replace nth column

I need to be able to search for a string in the first column and if that string exists than replace the nth column with "-9.99". AW12000012012 2.38 1.51 3.01 1.66 0.90 0.91 1.22 0.82 0.57 1.67 2.31 3.63 0.00 AW12000012013 1.52 0.90 1.20 1.34 1.21 0.67 ... (14 Replies)
Discussion started by: ncwxpanther
14 Replies

6. Shell Programming and Scripting

Need help with awk statement to break nth column in csv file into 3 separate columns

Hello Members, I have a csv file in the format below. Need help with awk statement to break nth column into 3 separate columns and export the changes to new file. input file --> file.csv cat file.csv|less "product/fruit/mango","location/asia/india","type/alphonso" need output in... (2 Replies)
Discussion started by: awk-admirer
2 Replies

7. Shell Programming and Scripting

Delete until Nth occurence (sed, awk)

Hello people, Once more I need your help with SED/AWK I need to delete up to the Nth occurence of a char (from the beggining) and until the Mth occurence of a char (from the end) Example: Input: a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z Output: i,j Must delete up to the... (2 Replies)
Discussion started by: drbiloukos
2 Replies

8. Shell Programming and Scripting

Calculating average for every Nth line in the Nth column

Is there an awk script that can easily perform the following operation? I have a data file that is in the format of 1944-12,5.6 1945-01,9.8 1945-02,6.7 1945-03,9.3 1945-04,5.9 1945-05,0.7 1945-06,0.0 1945-07,0.0 1945-08,0.0 1945-09,0.0 1945-10,0.2 1945-11,10.5 1945-12,22.3... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

9. Shell Programming and Scripting

Using AWK to find top Nth values in Nth column

I have an awk script to find the maximum value of the 2nd column of a 2 column datafile, but I need to find the top 5 maximum values of the 2nd column. Here is the script that works for the maximum value. awk 'BEGIN { subjectmax=$1 ; max=0} $2 >= max {subjectmax=$1 ; max=$2} END {print... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

10. Shell Programming and Scripting

Delete first block of text with sed/awk

Hello, guys! "filename" has blocks with three lines each in this fashion: 93909286 #verified has one bug 10909286 #unverified pending 10909286 #unverified pendingThe above example has duplicate blocks, and I have tried using sed to remove just one block... The... (2 Replies)
Discussion started by: teresaejunior
2 Replies
Login or Register to Ask a Question