Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove carriage returns from awk output

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove carriage returns from awk output
# 1  
Old 07-13-2017
Remove carriage returns from awk output
I'm on Linux version 2.6.32-696.3.1.el6.x86_64, using the Ksh shell.

I'm working with the input file:

Code:
John Daggett, 341 King Road, Plymouth MA
Alice Ford, 22 East Broadway, Richmond VA
Orville Thomas, 11345 Oak Bridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, Beaver Falls PA
Eric Adams, 20 Post Road, Sudbury MA
Hubert Sims, 328A Brook Road, Roanoke VA
Amy Wilde, 334 Bayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, Boston MA

My sed script "nameState" changes the 2-character state abbreviation to the full state name:
Code:
s/ CA/, California/
s/ MA/, Massachusetts/
s/ OK/, Oklahoma/
s/ PA/, Pennsylvania/
s/ VA/, Virginia/

I issued the sed and awk command within the command shell in emacs:
Code:
sed -f nameState list | awk -F, '{ print $4 ", " $0 }' > output

when I edit this file in Emacs I see:
Code:
Massachusetts^M, John Daggett, 341 King Road, Plymouth, Massachusetts
 Virginia^M, Alice Ford, 22 East Broadway, Richmond, Virginia
 Oklahoma^M, Orville Thomas, 11345 Oak Bridge Road, Tulsa, Oklahoma
 Pennsylvania^M, Terry Kalkas, 402 Lans Road, Beaver Falls, Pennsylvania
 Massachusetts^M, Eric Adams, 20 Post Road, Sudbury, Massachusetts
 Virginia^M, Hubert Sims, 328A Brook Road, Roanoke, Virginia
 California^M, Amy Wilde, 334 Bayshore Pkwy, Mountain View, California
 Massachusetts^M, Sal Carpenter, 73 6th Street, Boston, Massachusetts

which seems to be the output I want, except that it includes a ^M (carriage return?)

but when I issue the command cat output, I see:-
Code:
, John Daggett, 341 King Road, Plymouth, Massachusetts
, Alice Ford, 22 East Broadway, Richmond, Virginia
, Orville Thomas, 11345 Oak Bridge Road, Tulsa, Oklahoma
, Terry Kalkas, 402 Lans Road, Beaver Falls, Pennsylvania
, Eric Adams, 20 Post Road, Sudbury, Massachusetts
, Hubert Sims, 328A Brook Road, Roanoke, Virginia
, Amy Wilde, 334 Bayshore Pkwy, Mountain View, California
, Sal Carpenter, 73 6th Street, Boston, Massachusetts

Why am I not seeing the state in the first field of this file? How can I get rid of the ^M carriage return that the awk command seems to insert?

Thanks for your help!


Moderator's Comments:
Mod Comment
Please wrap all code, files, input & output/errors in CODE tags.
It makes it easier to read and preserves multiple spaces for indenting or fixed width data.
Last edited by rbatte1; 07-13-2017 at 01:31 PM.. Reason: Added CODE/ICODE tags & tightened up line spaces to save blank lines.
# 2  
Old 07-13-2017
Code:
dos2ux < list | sed -f nameState | awk -F, '{ print $4 ", " $0 }' > output

# 3  
Old 07-13-2017
Thank you for your solution and for advice on good practices in formatting posts.

At my installation, the utility is called dos2unix
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

TR not removing carriage returns

I have a CSV with carriage returns in place of newlines. I am trying to use tr to remove them, but it isn't working. Academic year,Term,Course name,Period,Last name,Nickname 2012-2013,First Semester,English 12,4th Period,Arnold,Adam 2012-2013,First Semester,English 12,4th Period,Adams,Jim... (1 Reply)
Discussion started by: nextyoyoma
1 Replies

2. Shell Programming and Scripting

Awk to remove carriage return from 65th field

Hi, I have a pipe delimited file. There are around 700 columns in the file. The 65th column has carriage return which is causing read issue with our ETL process. I would like to replace the new line characters in 65th field with "nothing" i have return the following code and need help to... (7 Replies)
Discussion started by: pinnacle
7 Replies

3. Shell Programming and Scripting

for loop returns more output with awk

I need to get total number of hdisk not assigned to any VGs. PDC # lspv |grep None |awk '{print $1}' |wc 131 131 1099 So, it shows 131 hdisks. I need to look at the individual hdisk fget_config info like below: PDC # fget_config -Av |grep hdisk230 hdisk230 dac1 229... (4 Replies)
Discussion started by: Daniel Gate
4 Replies

4. Emergency UNIX and Linux Support

Adding carriage returns to file using sed/awk

Hello, I need help adding carriage returns at specific intervals (say 692 characters) to a text file that's one continous string. I'm working in AIX5.3. Any quick help is appreciated. Thanks! (2 Replies)
Discussion started by: bd_joy
2 Replies

5. Shell Programming and Scripting

removing carriage returns in text file

Hi I have a text file that looks like this: A B C D E F G H I I want it to be reformatted to A;B;C; D;E;F; G;H;I; (4 Replies)
Discussion started by: coolnfunky
4 Replies

6. Shell Programming and Scripting

Replacing Carriage returns without loosing EOL

Hello, I have read a few threads on this subject and tried a few things out, but still come up short. There was one good example, then the last reply was something to the effect of 'Use Sed' & 'Read a book'... Well I read a bunch of online tutorials on sed, awk, tr, but still can't get the... (2 Replies)
Discussion started by: Majiktom
2 Replies

7. Shell Programming and Scripting

removing thousand of carriage returns using sed

I need to replace thousands of carriage returns/line breaks in a large xml file and with spaces. I hope to do so with a script, called, for example, "removeCRs." I would invoke this at the command line as ml5003$ sed -f /Users/ml5003/removeCRs oldFile > newFile The script, I presume, would... (4 Replies)
Discussion started by: ml5003
4 Replies

8. Shell Programming and Scripting

Removing carriage returns with sed

How do we delete all carriage returns after a particular string using sed inside a K Shell? e.g. I have a text file named file1 below: $ more file1 Group#=1 User=A Role=a1 Group#=2 User=B Role=a1 Role=b1 Group#=3 User=C Role=b1 I want the carriage returns to be delete on the... (12 Replies)
Discussion started by: stevefox
12 Replies

9. Shell Programming and Scripting

Remove Carriage returns between strings in a field

Is there any way to remove carriage retuns between the records? We have input records separated by TABS and have carriage returns as below: 123 456 789 ABC "1952.00" 678 "abcdef ghik lmno" Above we... (10 Replies)
Discussion started by: acheepi
10 Replies

10. Shell Programming and Scripting

spaces and carriage returns in 'here documents'

As the title suggests, i am having some trouble figuring out how to pass spaces and carriage returns to a 'here document' ie #!/bin/bash /usr/local/install_script.sh <<SCRIPT yes no <pass carriage retun here> yes no <pass a space and then a carriage return here> exit SCRIPT any... (0 Replies)
Discussion started by: hcclnoodles
0 Replies
Login or Register to Ask a Question