Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Remove character \r and \n in awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Remove character \r and \n in awk
# 1  
Old 07-06-2019
Remove character \r and \n in awk
Hi Everybody:


I need your help, please...
I have this file *.txt


Code:
 0000 | 16010201 22000000 67892000 00000000 00000000 00000100 72246681 28E08236
      | ~~~~"~~~g~ ~~~~~~~~~~~~~r$f~(~~6 |
 0020 | 10476173 90010100 10000000 00000001 05000226 17163011 12442212 48140484
      | ~Gas~~~~~~~~~~~~~~~&~~0~~D"~H~~~ |
 0040 | 05100001 00064555 50224761 73900101 0010D221 22211143 844489F6 F0F5F7F1
      | ~~~~~~EUP"Gas~~~~~~!"!~C~D~~~~~~ |
 0060 | F7F1F1F1 F2F4F4F7 F1F5F8F0 F0F0F6F6 F6F5F9F6 F4F34040 40404040 4040E3C5
      | ~~~~~~~~~~~~~~~~~~~~~~@@@@@@@@~~ |
 0080 | D3D4C5E7 40C7D7D6 D540D7E9 C140C3C1 D9E2D640 404040D4 C5E7C9C3 D640C4C6
      | ~~~~@~~~~@~~~@~~~~~@@@@~~~~~~@~~ |
 00A0 | 40404040 D4E70484 59010056 5F2A0204 8482025C 00950500 80008000 9A031602
      | @@@@~~~~Y~~V_*~~~~~\~~~~~~~~~~~~ |
 00C0 | 269C0100 9F020600 00000105 009F0306 00000000 00009F10 0706010A 03A08000
      | &~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
 00E0 | 9F1A0204 849F2608 15CDE0B9 2E70B84B 9F3303E0 B0C89F36 0200139F 3704711B
      | ~~~~~~&~~~~~.p~K~3~~~~~6~~~~7~q~ |
 0100 | FCA60AF0 F0F0F0F0 F0F6F5F9 F9040500 00100980 00000000 000000E8 05800000
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
 0120 | 0000
      | ~~                               |
      
 0000 | 16010200 AB678920 00000001 10004B8C 101B0192 B2000110 72202281 0ED08206
      | ~~~~~g~ ~~~~~~K~~~~~~~~~r "~~~~~ |
 0020 | 10476173 90010100 10000000 00000001 05000226 17163011 12440484 00010006
      | ~Gas~~~~~~~~~~~~~~~&~~0~~D~~~~~~ |
 0040 | 455550F6 F0F5F7F1 F7F1F1F1 F2F4F4F0 F9F9F7F4 F2F0F0F7 F1F5F8F0 F0F0F6F6
      | EUP~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
 0060 | F6F5F9F6 F4F34040 40404040 404009F4 40404040 404040F2 04841401 00119F36
      | ~~~~~~@@@@@@@@~~@@@@@@@~~~~~~~~6 |
 0080 | 02001391 0AD690C2 A90233B0 F4303015 E0000000 00000000 D5030605 76224000
      | ~~~~~~~~~~3~~00~~~~~~~~~~~~~v"@~ |
 00A0 | 18D5D740 40058000 000002
      | ~~~@@~~~~~~                      |


They are only two records, I need every record in one line:

Code:
1601020122000000678920000000000000000000000001007224668128E082361047617390010100100000000000000105000226171630111244221248140484051000010006455550224761739001010010D22122211143844489F6F0F5F7F1F7F1F1F1F2F4F4F7F1F5F8F0F0F0F6F6F6F5F9F6F4F34040404040404040E3C5D3D4C5E740C7D7D6D540D7E9C140C3C1D9E2D640404040D4C5E7C9C3D640C4C640404040D4E70484590100565F2A02048482025C00950500800080009A031602269C01009F02060000000105009F03060000000000009F100706010A03A080009F1A0204849F260815CDE0B92E70B84B9F3303E0B0C89F360200139F3704711BFCA60AF0F0F0F0F0F0F6F5F9F90405000010098000000000000000E8058000000000
16010200AB6789200000000110004B8C101B0192B2000110722022810ED082061047617390010100100000000000000105000226171630111244048400010006455550F6F0F5F7F1F7F1F1F1F2F4F4F0F9F9F7F4F2F0F0F7F1F5F8F0F0F0F6F6F6F5F9F6F4F3404040404040404009F440404040404040F20484140100119F36

My program give that as result:


Code:
1601020122000000678920000000000000000000000001007224668128E08236
1047617390010100100000000000000105000226171630111244221248140484
051000010006455550224761739001010010D22122211143844489F6F0F5F7F1
F7F1F1F1F2F4F4F7F1F5F8F0F0F0F6F6F6F5F9F6F4F34040404040404040E3C5
D3D4C5E740C7D7D6D540D7E9C140C3C1D9E2D640404040D4C5E7C9C3D640C4C6
40404040D4E70484590100565F2A02048482025C00950500800080009A031602
269C01009F02060000000105009F03060000000000009F100706010A03A08000
9F1A0204849F260815CDE0B92E70B84B9F3303E0B0C89F360200139F3704711B
FCA60AF0F0F0F0F0F0F6F5F9F90405000010098000000000000000E805800000
0000
16010200AB6789200000000110004B8C101B0192B2000110722022810ED08206
1047617390010100100000000000000105000226171630111244048400010006
455550F6F0F5F7F1F7F1F1F1F2F4F4F0F9F9F7F4F2F0F0F7F1F5F8F0F0F0F6F6
F6F5F9F6F4F3404040404040404009F440404040404040F20484140100119F36
020013910AD690C2A90233B0F4303015E000000000000000D503060576224000
18D5D74040058000000002



Awk program


Code:
BEGIN{start=0;}
{
     get_traza();
 }
 END{}

function get_traza() {
if ($1=="0000" || getline==0) {
    if (NR != 1)
      print msj;
    msj="";
    start=1;
}

if (start=1) {
    #p NR;
    if ($1 != "|") {
        k = 3;
        while (k<=10) {
            msj = msj $k;
            k++;
        }

    }
}
 }


How do I revove the character '\r' and/or '\n'?, when I debug I get this:

Code:
msj = "1601020122000000678920000000000000000000000001007224668128E08236\r"


Can you help me, please?


Thanks
# 2  
Old 07-06-2019
Hi, you can remove the newlines by specifying ORS="" in the BEGIN section, which will stop awk from printing a newline character after every record, so the output becomes all one line.
Unix files should have a closing newline, so you would need to print one in the END section:
Code:
printf "\n"


If there are \r characters then most likely your input file is not in the proper format and you need to remove them first:

Code:
tr -d '\r' < infile > outfile


--
note:
To test if the variable start is equal to 1 , you need to use two equal signs if (start==1)

also note:
getline will return 0 in case of EOF, but -1 in case of an error, so instead of using ==0 it better to use <=0 or !=1
Last edited by Scrutinizer; 07-06-2019 at 03:36 AM..
# 3  
Old 07-06-2019
Within awk, you can remove the \r with sub (/\r$/, "").



Try also
Code:
awk -F\| '
                {gsub (/ |\r$/, "")
                }
$1 == "0000"    {printf LE
                 LE = ORS
                }
$1 != ""        {printf "%s", $2
                }
END             {printf ORS
                }
' file
1601020122000000678920000000000000000000000001007224668128E082361047617390010100100000000000000105000226171630111244221248140484051000010006455550224761739001...
16010200AB6789200000000110004B8C101B0192B2000110722022810ED082061047617390010100100000000000000105000226171630111244048400010006455550F6F0F5F7F1F7F1F1F1F2F4F4...

Be aware that your desired output does NOT reflect the result of applying your logics to the sample input.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies

2. Shell Programming and Scripting

How to remove certain character strings with awk?

Hi all, I need to remove DBPATH= and /db from the string below using awk (or sed, as it also exists on the machine). Input: DBPATH=/some/path/database/db Desired output: /some/path/database Thank you! (8 Replies)
Discussion started by: ejianu
8 Replies

3. Shell Programming and Scripting

Want to remove / and character using awk or sed

Below i am trying to remove "/" and "r" from the output, so i need output as: hdiskpower3 hdisk0 hdisk1 #inq | grep 5773 | awk '{print $1}' | sed 's/dev//g' | awk -F"/" '{$1=$1}1' .....................................................//rhdiskpower0 //rhdiskpower1 //rhdiskpower2... (3 Replies)
Discussion started by: aix_admin_007
3 Replies

4. UNIX for Dummies Questions & Answers

Remove last character in each line

Hi guys, Does anyone know how to remove the last character in each of the line? This is what I have: ABCDE.1 GLSJD.2 HIJPL.2 HKAGB.3 IUBWQ.1 What I want (remove the dot and number): ABCDE GLSJD HIJPL HKAGB IUBWQ I tried to use this: sed 's/.*//' But I'm not sure if that is... (3 Replies)
Discussion started by: narachaid
3 Replies

5. Shell Programming and Scripting

How to remove the ^@ character in a file.

I receive a file which is Dos format. When I view the file using vi, I was able to find ^M, ^@ characters in beteen records. I wanted to remove these control/special characters. I used the dos2unix command. This removed the ^M characters. But I am unable to remove the ^@ character. I tried even the... (6 Replies)
Discussion started by: machomaddy
6 Replies

6. Shell Programming and Scripting

any savant ? using AWK/SED to remove newline character between two strings : conditional removal

I'd like to remove (do a pattern or precise replacement - this I can handle in SED using Regex ) ---AFTER THE 1ST Occurrence ( i.e. on the 2nd occurrence - from the 2nd to fourth occurance ) of a specific string : type 1 -- After the 1st occurrence of 1 string1 till the 1st occurrence of... (4 Replies)
Discussion started by: sieger007
4 Replies

7. HP-UX

How to remove new line character and append new line character in a file?

Hi Experts, I have data coming in 4 columns and there are new line characters \n in between the data. I need to remove the new line characters in the middle of the row and keep the \n character at the end of the line. File is comma (,) seperated. Eg: ID,Client ,SNo,Rank 37,Airtel \n... (8 Replies)
Discussion started by: sasikari
8 Replies

8. UNIX for Dummies Questions & Answers

How to remove \ character

Dear Members, I have a file which is a single line file. It has "\" character and i need to replace this character with a new line character. How can we do this? I tried with sed but it did not work. sed 's//"\n"/g' t1 > t2Thanks Sandeep (3 Replies)
Discussion started by: sandeep_1105
3 Replies

9. Shell Programming and Scripting

Remove a ^M character

Hi, I'd like to ask for some help with the following: I've cut a couple of columns of file1 to create file2 with the following code: cur -f 1,3,8 file1 > file2 Then I need to transfer file 2 from UNIX to Windows and use it further. Unfortunatelly, for some reason the line is displayed... (4 Replies)
Discussion started by: zajtat
4 Replies

10. Shell Programming and Scripting

Remove last character of a term

Hi All, I have a few terms with a comma as the last character. Can any experts show me how to remove the last character? Note the the length of the term is not fix. Input: 1, 2, 12, 14, 103, 198, 3006, Output, 1 (6 Replies)
Discussion started by: Raynon
6 Replies
Login or Register to Ask a Question