Visit The New, Modern Unix Linux Community


Help with awk or sed Command to Replace Text in Files


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Help with awk or sed Command to Replace Text in Files
# 1  
Help with awk or sed Command to Replace Text in Files

Hello Everyone,

I have many files like so:
Code:
file1.txt
file2.txt
file3.txt

Within each file I have many lines of random text separated by commas like so:
Code:
abcAAA,123,defAA,456777,ghiA,789
jklB,101,mnoBBB,11211,pqrB,13111
stuCC,415,vwxCCCC,161,yzaC,718

I am trying to use SED or AWK to replace the 4th column with XXX on each line in each file so the result would be:

The same files like so:
Code:
file1.txt
file2.txt
file3.txt

Now within each file:
Code:
abcAAA,123,defAA,XXX,ghiA,789
jklB,101,mnoBBB,XXX,pqrB,13111
stuCC,415,vwxCCCC,XXX,yzaC,718


I have a workaround that I've come up with but it combines all the files into one file, unfortunately. (I need the files to be edited in place)
Here is my attempt:
Code:
cat *.txt | awk -F, '{print $1 "," $2 "," $3 ",XXX," $5 "," $6}' > result_file.txt


Last edited by D3U5X; 01-28-2020 at 04:53 PM..
# 2  
One approach is to use a for loop to open one file at a time, modify and redirect the output to a temporary file, rename the temporary file back to original file:-
Code:
for file in *.txt; do awk -F, '{$4="XXX"}1' OFS=, $file > tmp; mv tmp $file; done

This User Gave Thanks to Yoda For This Post:
# 3  
Thank you, Yoda! I will give this a try and update this thread as solved if I can get it working. Smilie

--- Post updated at 11:08 PM ---

In my initial tests the solution is almost perfect but I am getting an extra line in my files:
Code:
sdf,asdf,XXX,asdf,asdf
sdf,asdf,XXX,asdf,asdf
sdf,asdf,XXX,asdf,asdf
sdf,asdf,XXX,asdf,asdf
,,XXX

I am working out why this is and I will update.
# 4  
Looks like you have a trailing empty line in your file. And, you seem to replace $3 now, not $4.
This User Gave Thanks to RudiC For This Post:
# 5  
Correct! Getting rid of the extra line fixed it. I was playing around with different column positioning that's why I had 3 instead of 4.

Thank you all! I will now mark this as solved!

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #844
Difficulty: Medium
Bittorent, Gnutella and other distributed file transfers systems use do not user peer-to-peer, by default.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Using sed or awk to replace digits in files

Hello; I am not good at file and stream editing. I need to replace a few digits in two files. The lines in files looks like this: Line in the first file, /dw300/data/obe/2019273.L800JR.1909.273 Line in second file, 1|2019273.L800JR.1909.273 I will write a function to connect to... (7 Replies)
Discussion started by: duke0001
7 Replies

2. Debian

Using awk and sed to replace text

Good Day Every one I have a problem finding and replacing text in some large files that will take a long time to manually edit. Example text file looks like this #Example Large Text File unix linux dos squid bind dance bike car plane What im trying to do is to edit all the... (4 Replies)
Discussion started by: linuxjunkie
4 Replies

3. Shell Programming and Scripting

Need command to replace empty using sed/awk

Hi, In a file we have the following data like as below abcdef="cfg-1-15" bmmdda-g-45-2 yhdiao"rtg-1-df-34" I need a sed/awk command to replace the above string with empty. Thx, (1 Reply)
Discussion started by: kirankumar
1 Replies

4. Shell Programming and Scripting

awk command to replace columns in 2 files

Hi All, I already have a code which replaces column 14 of NPBR.XTR.tmp with column 8 of NPBR3G.XTR.final awk -F'\|' 'FNR==NR{a= $2"^"$8;next;}a{split(a,b,"^");$8=b;$14=b;}1' OFS="|" ${SHTEMP}NPBR3G.XTR.final ${SHTEMP}NPBR.XTR.tmp > ${SHTEMP}NPBR.XTR.final I also need to replace column 15... (2 Replies)
Discussion started by: nua7
2 Replies

5. Shell Programming and Scripting

Sed/awk/perl command to replace pattern in multiple lines

Hi I know sed and awk has options to give range of line numbers, but I need to replace pattern in specific lines Something like sed -e '1s,14s,26s/pattern/new pattern/' file name Can somebody help me in this.... I am fine with see/awk/perl Thank you in advance (9 Replies)
Discussion started by: dani777
9 Replies

6. Shell Programming and Scripting

Sed command to replace with pattern except for text and closing parentheses

Can someone help me with a sed command: There will be multiple occurences in a file that look like this: MyFunction(12c34r5) and I need to replace that with just the 12c34r5 for every occurrence. The text between the parentheses will be different on each occurrence, so I can't search for that.... (4 Replies)
Discussion started by: missb
4 Replies

7. Shell Programming and Scripting

using sed/awk to replace a block of text in a file?

My apologies if this has been answered in a previous post. I've been doing a lot of searching, but I haven't been able to find what I was looking for. Specifically, I am wondering if I can utilize sed and/or awk to locate two strings in a file, and replace everything between those two strings... (12 Replies)
Discussion started by: kiddsupreme
12 Replies

8. Shell Programming and Scripting

Extraction of text using sed or awk command

Hi All, I need to extract 543 from the command below : # pvscan PV /dev/sdb1 VG vg0 lvm2 Total: 1 543.88 GB] / in use: 1 / in no VG: 0 I have the following command which does the job, but I think this could be achieved in a more simple way using sed or awk. Any help is... (7 Replies)
Discussion started by: nua7
7 Replies

9. Shell Programming and Scripting

How to replace a range of text with sed or awk?

Howdy! I'm trying to automate editing of a configuration file (custom.conf for GDM). I need to find every line between a line that starts with "" and the next line that starts with "", I want to preserve that line, but then delete all the lines in that configuration section and then insert... (3 Replies)
Discussion started by: TXTad
3 Replies

10. Shell Programming and Scripting

sed or awk to convert text files with recurring headings to rows and colum

I have many text file reports generated by a Information Assurance tool that I need to get into a .CSV format or Excel tab delimited format. I want to use sed or awk to grab all the information in the sample text file below and create column headings:Risk ID, Risk Level, Category, Description, How... (5 Replies)
Discussion started by: Bjoeboo
5 Replies

Featured Tech Videos