Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Modify text file using sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Modify text file using sed
# 1  
Old 04-01-2014
RedHat Modify text file using sed
Hello all,

I have some text files I need to do the following on:

Delete banner page (lines 1-56) --I am doing this using sed
Remove ^M --I am doing this using vi
Remove trailer page --this can vary based on the contents of the file, it usually starts with ***************************

I am able to do the first two steps manually by just issuing the commands against the filename, but I am wondering if there is a systematic approach that I can use to make this happen all in one simple script.
# 2  
Old 04-01-2014
^M seem to be of DOS origin in which case you should convert the file first using dos2ux or dos2unix depending of your ux flavor
# 3  
Old 04-01-2014
Systematic approach is a good idea, the awk language can handle a lot of this in one invocation.

Lots of people don't have dos2unix but you don't even need it. awk or tr can do it.

Code:
awk '{ sub(/\r/, ""); } # Delete carriage returns
NR <= 56 { next } # Don't print lines from banner page
# Quit once you find ***********
/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
# Otherwise, print all lines
{ print }' < inputfile > outputfile

This User Gave Thanks to Corona688 For This Post:
jacobs.smith
# 4  
Old 04-01-2014
Thank you for steering me in the right direction, I think I am almost there. My goal is to take a file, "massage" it, and output the contents. I am expecting the below to massage JEFFTEST files and output JEFFTEST.TIMESTAMP (Whatever that may be)

Code:
find /opt/test -type f -name "JEFFTEST" | while read name

do
	timestamp=$(date +%s)
	awk '{ sub(/\r/, ""); } # Delete carriage returns
	NR <= 56 { next } # Don't print lines from banner page
	# Quit once you find ***********
	/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
	# Otherwise, print all lines
	{ print }' < $name > $name.$timestamp
	
done

# 5  
Old 04-01-2014
Test a single file before you test thousands of files.
# 6  
Old 04-01-2014
A bit more efficient and robust:
Code:
timestamp=$(date +%s)
find /opt/test -type f -name "JEFFTEST" |
while read name
do
   awk '
    NR <= 56 { next } # Don't print lines from banner page
    { sub(/\r/$, ""); } # Delete carriage return at the end of the lines
    # Quit once you find ***********
    /\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ { exit }
    # Otherwise, print all lines
    { print }
   ' < "$name" > "$name.$timestamp"
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Modify text file if found multiple pattern match for every line.

Looking for help, i have input file like below and want to modify to expected output, if can without create additional file, hope can direct modify it. have 2 thing need do. 1st is adding a word (testplan generation off) after ! ! IPG: Tue Aug 07 14:31:17 2018 2nd is adding... (16 Replies)
Discussion started by: kttan
16 Replies

2. Shell Programming and Scripting

Modify text file using awk

I have text file with lines as shown here. Each row has 11 columns separated by tab. In each row, i want to split the 8th column such that the output should look like shown below. Here value in the 9th column is DP value and in the 10th column is MQ value followed by the values after resource.EFF=.... (15 Replies)
Discussion started by: mehar
15 Replies

3. Shell Programming and Scripting

Modify one line in a plain text file

Hi everyone, I want to know, if there is a way to modify one line in a text file with unix script, with out re-writing all the file. For example, i have this file: CONFIGURATION_1=XXXX CONFIGURATION_2=YYYY CONFIGURATION_3=ZZZZ supose i have a command or function "modify" that... (7 Replies)
Discussion started by: Xedrox
7 Replies

4. Shell Programming and Scripting

Modify the text file by script

Hi All the Helpers! I have a text file which looks like input.txt.I would request to please suggest me how can I make this file look like output.txt input.txt VOP 111 0 1 2 DEM 111 0 222 333 444 555 879 888 987 888 989 VOP 118 0... (2 Replies)
Discussion started by: Indra2011
2 Replies

5. Shell Programming and Scripting

Modify the file with awk,sed or perl

Hi All, I need help from any of you.Would be so thankful for your help. I/P DDDD,1045,161,1557,429,1694,800,1911,1113,2460,1457,2917> 1609,3113,1869,3317,2732,3701,3727,4132,5857,5107> 9004,6496 DDDD,1125,157,1558,429,1694,800,1911,1117,2432,1444,2906>... (2 Replies)
Discussion started by: Indra2011
2 Replies

6. Shell Programming and Scripting

Modify text file using shell script

Hi, I have a text file which is following format - COL VAL ABC 1 ABC 2 ABC 3 ABC 4 ABC 5 My requirement is to search for a particular value (provided by user) in the file and comment the previous entries including that as well. E.g. If I search for number 3, then the output... (6 Replies)
Discussion started by: bhupinder08
6 Replies

7. UNIX for Dummies Questions & Answers

Modify Text File

Hi, I would like to remove any lines from a text file that begin with #, or that are blank. How can I do that with BASH? Mike (3 Replies)
Discussion started by: msb65
3 Replies

8. Shell Programming and Scripting

Need help to modify perl script: Text file with line and more than 1 space

Dear Friends, I am beginner in Perl and trying to find the problem in a script. Kindly help me to modify the script. My script is not giving the output for the last field and followed text (LA: Language English). Input file & script as follows: Input file: Thu Mar 19 2:34:14 EDT 2009 STC... (3 Replies)
Discussion started by: srsahu75
3 Replies

9. Shell Programming and Scripting

Modify Specific Line of a Text File

Given a text file, how do you add a line of text after a specific line number? I believe I would want to use "sed" but I am unsure of the syntax. Thank you. Mike (5 Replies)
Discussion started by: msb65
5 Replies

10. Shell Programming and Scripting

Modify a text or xml file

Hi all, I want to write a shell which would allow me to edit a text file or a xml file. Basically i want to add a new node in a existing xml file. The values for this new node are based on user input. Thanks in advance Zing (9 Replies)
Discussion started by: zing
9 Replies
Login or Register to Ask a Question