Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Tip to remove line endings and spaces on a pre-formatted text file? Post 303041486 by newbie_01 on Monday 25th of November 2019 03:24:33 PM
Old 11-25-2019
Tip to remove line endings and spaces on a pre-formatted text file?

Hi,

At the moment, using Notepad++ to do a search and replace, manually section by section which is real painful. Yeah, so copying each section of the line of text and putting into a file and then search and replace, need at least 3-operations in Notepad++.

Here's hoping I will be able to get some help on how to do this via some script or sed/awk combination maybe?

Basically, I have an Oracle tnsnames,.ora file that contains several section of text like below. So, there will be lots of the following, I just listed two of them at the moment.

Code:
test123.abcde.com.ny =
 (DESCRIPTION=
   (LOAD_BALANCE=off)(CONNECT_TIMEOUT=10)(TRANSPORT_CONNECT_TIMEOUT=6)
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan2.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dca.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dcb.abcde.com.ny)(PORT=1521)))
   (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=abcq1_app.abcde.com.ny))
 )
test124.abcde.com.ny =
 (DESCRIPTION=
   (LOAD_BALANCE=off)(CONNECT_TIMEOUT=10)(TRANSPORT_CONNECT_TIMEOUT=6)
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan2.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dca.abcde.com.ny)(PORT=1521)))
   (ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dcb.abcde.com.ny)(PORT=1521)))
   (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=abcq1_app.abcde.com.ny))
 )

I need to convert it to so that it rids itself of the spaces and line endings so that it now looks as below:


Code:
test123.abcde.com.ny =
(DESCRIPTION=(LOAD_BALANCE=off)(CONNECT_TIMEOUT=10)(TRANSPORT_CONNECT_TIMEOUT=6)(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan2.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dca.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dcb.abcde.com.ny)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=abcq1_app.abcde.com.ny)))
test124.abcde.com.ny =
(DESCRIPTION=(LOAD_BALANCE=off)(CONNECT_TIMEOUT=10)(TRANSPORT_CONNECT_TIMEOUT=6)(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan2.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dca.abcde.com.ny)(PORT=1521)))(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=abcq1-scan1-dcb.abcde.com.ny)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=abcq1_app.abcde.com.ny)))

Please advise. Thanks.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to remove FIRST Line of huge text file on Solaris

i need help..!!!! i have one big text file estimate data file size 50 - 100GB with 70 Mega Rows. on OS SUN Solaris version 8 How i can remove first line of the text file. Please suggest me for solutions. Thank you very much in advance:) (5 Replies)
Discussion started by: madoatz
5 Replies

2. Shell Programming and Scripting

Adding specific text and spaces to each line in a text file

Hi, I wanted to add specific text to each row in a text file containing three rows. Example: 0 8 7 6 5 5 7 8 9 0 7 9 7 8 9 0 1 2 And I want to add a 21 at the beginning of the first row, and blank spaces at the beginning of the second two rows. To get this: 21 0 8 7 6 5 5 7 8... (4 Replies)
Discussion started by: hertingm
4 Replies

3. Shell Programming and Scripting

Remove spaces from first field, and write entire contents into other text file

Hi all, I have searched and found various threads about removing spaces from a field within a text file. Unfortunately, I have not found exactly what I'm looking for, nor am I adept enough to modify what I've found into what I need. I use the following command to remove the first line... (3 Replies)
Discussion started by: carriehoff
3 Replies

4. Shell Programming and Scripting

Remove blank spaces in a text file...

Hi, I have this problem that there are blank spaces in my text file... i want to remove them line 1 line 2 line 3 I want to remove the space between line 2 and line 3... I tried sed... it work but it prints the whole text file at the command prompt which i dont want.... sde i tried was... (4 Replies)
Discussion started by: bhagya2340
4 Replies

5. Shell Programming and Scripting

Need to remove improperly formatted fortran output line from files, tried sed

I have been trying to remove some improperly formatted lines of output from fortran code I have been using. The problem is that I have some singularities in the math for some points that causes an incorrectly large value to be reported that exceeds the normal formating set in the code resulting in... (2 Replies)
Discussion started by: gillesc_mac
2 Replies

6. Shell Programming and Scripting

sed to remove 1st two characters every line of text file

what is the sed command to remove the first two characters of every line of a text file? each line of the text file has the same amount of characters, and they are ALL NUMERIC. there are hundreds of lines though. for example, >cat file1.txt 10081551 10081599 10082234 10082259 20081134... (20 Replies)
Discussion started by: ajp7701
20 Replies

7. Shell Programming and Scripting

How to remove blank line from a text file?

Hi All, I am creating a text file using perl. The first record I am writing as "$line" and all the other as "\n$line". At the end the file is having N number of lines. I am using this file for MLOAD (Teradata), which is reading N+1 lines in the file and failing.I am not able to find new line... (2 Replies)
Discussion started by: unankix
2 Replies

8. Shell Programming and Scripting

How Select numbers from a line of text, and remove leading spaces?

I have a text file with a line of text that contains numbers and text formatted into groups. I need to extract the number that can be either 1,2 or 3 digits long. Then write it to a variable, but i need to remove any leading spaces in the number first. I can get the numbers out but how to remove... (12 Replies)
Discussion started by: kcpoole
12 Replies

9. UNIX Desktop Questions & Answers

To remove the extra spaces at the end of each line in a file

I have a file of about 10k records and eace line is having an extra space of 5 byte at the end.. Iwant to remove the extra spaces at the end of each line.. Can someone please help me out.. I tried using sed command and its not working... can someone please help me out. (3 Replies)
Discussion started by: rammohan
3 Replies

10. Shell Programming and Scripting

How to remove spaces on a line?

Hi, suppose I have the following data: albert music=top40 age=20 bob music=punk rock age=25 candy music=r n b age=22 dave music=mozart or bach only age=30 I want to extract and manipulate the music column but it's got spaces in it. How can I substitute the space with an underscore... (2 Replies)
Discussion started by: almonds
2 Replies
dos2unix(1)							    2010-04-03							       dos2unix(1)

NAME
dos2unix - DOS/MAC to UNIX and vice versa text file format converter SYNOPSIS
dos2unix [options] [-c CONVMODE] [-o FILE ...] [-n INFILE OUTFILE ...] unix2dos [options] [-c CONVMODE] [-o FILE ...] [-n INFILE OUTFILE ...] DESCRIPTION
The Dos2unix package includes utilities "dos2unix" and "unix2dos" to convert plain text files in DOS or MAC format to UNIX format and vice versa. Binary files and non-regular files, such as soft links, are automatically skipped, unless conversion is forced. Dos2unix has a few conversion modes similar to dos2unix under SunOS/Solaris. In DOS/Windows text files line endings exist out of a combination of two characters: a Carriage Return (CR) followed by a Line Feed (LF). In Unix text files line endings exists out of a single Newline character which is equal to a DOS Line Feed (LF) character. In Mac text files, prior to Mac OS X, line endings exist out of a single Carriage Return character. Mac OS X is Unix based and has the same line endings as Unix. OPTIONS
-c, --convmode CONVMODE Set conversion mode. Where CONVMODE is one of: ascii, 7bit, iso, mac with ascii being the default. -f, --force Force conversion of all files. Also binary files. -h, --help Display online help. -k, --keepdate Keep the date stamp of output file same as input file. -L, --license Display software license. -l, --newline Add additional newline. dos2unix: Only DOS line endings are changed to two Unix line endings. In Mac mode only Mac line endings are changed to two Unix line endings. unix2dos: Only Unix line endings are changed to two DOS line endings. In Mac mode Unix line endings are changed to two Mac line endings. -n, --newfile INFILE OUTFILE ... New file mode. Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be used or you WILL lose your files. -o, --oldfile FILE ... Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used. -q, --quiet Quiet mode. Suppress all warning and messages. -V, --version Display version information. CONVERSION MODES
Conversion modes ascii, 7bit, and iso are similar to those of dos2unix/unix2dos under SunOS/Solaris. ascii dos2unix: In this mode DOS line endings are converted to Unix line endings. Unix and Mac line endings are not changed. unix2dos: In this mode Unix line endings are converted to DOS line endings. DOS and Mac line endings are not changed. Although the name of this mode is ASCII, which is a 7 bit standard, the actual mode is 8 bit. mac dos2unix: In this mode Mac line endings are converted to Unix line endings. DOS and Unix line endigs are not changed. You can also use the command "mac2unix" to run dos2unix in Mac mode. unix2dos: In this mode Unix line endings are converted to Mac line endings. DOS and Mac line endigs are not changed. You can also use the command "unix2mac" to run unix2dos in Mac mode. 7bit In this mode DOS line endings are converted to Unix line endings or vice versa. All 8 bit non-ASCII characters (with values from 128 to 255) are converted to a space. iso In this mode DOS line endings are converted to Unix line endings or vice versa. Characters are converted between the DOS character set (code page) CP437 and ISO character set ISO-8859-1 on Unix. CP437 characters without ISO-8859-1 equivalent, for which conversion is not possible, are converted to a dot. The same counts for ISO-8859-1 characters without CP437 counterpart. CP437 is mainly used in the USA. In Western Europe CP850 is more standard. Another option to convert text files between different encodings is to use dos2unix in combination with iconv(1). Iconv can convert between a long list of character encodings. Some examples: Convert from DOS DOSLatinUS to Unix Latin-1 iconv -f CP437 -t ISO-8859-1 in.txt | dos2unix > out.txt Convert from DOS DOSLatin1 to Unix Latin-1 iconv -f CP850 -t ISO-8859-1 in.txt | dos2unix > out.txt Convert from Windows WinLatin1 to Unix Latin-1 iconv -f CP1252 -t ISO-8859-1 in.txt | dos2unix > out.txt Convert from Windows WinLatin1 to Unix UTF-8 (Unicode) iconv -f CP1252 -t UTF-8 in.txt | dos2unix > out.txt Convert from Windows UTF-16 (Unicode) to Unix UTF-8 (Unicode) iconv -f UTF-16 -t UTF-8 in.txt | dos2unix > out.txt Convert from Unix Latin-1 to DOS DOSLatinUS unix2dos < in.txt | iconv -f ISO-8859-1 -t CP437 > out.txt Convert from Unix Latin-1 to DOS DOSLatin1 unix2dos < in.txt | iconv -f ISO-8859-1 -t CP850 > out.txt Convert from Unix Latin-1 to Windows WinLatin1 unix2dos < in.txt | iconv -f ISO-8859-1 -t CP1252 > out.txt Convert from Unix UTF-8 (Unicode) to Windows WinLatin1 unix2dos < in.txt | iconv -f UTF-8 -t CP1252 in.txt > out.txt Convert from Unix UTF-8 (Unicode) to Windows UTF-16 (Unicode) unix2dos < in.txt | iconv -f UTF-8 -t UTF-16 > out.txt See also <http://czyborra.com/charsets/codepages.html> and <http://czyborra.com/charsets/iso8859.html>. UNICODE
Unicode files can be encoded in different encodings. On Unix/Linux Unicode files are mostly encoded in UTF-8 encoding. UTF-8 is ASCII compatible. UTF-8 files can be in DOS, Unix or Mac format. It is safe to run dos2unix/unix2dos on UTF-8 encoded files. On Windows mostly UTF-16 encoding is used for Unicode files. Dos2unix/unix2dos should not be run on UTF-16 files. UTF-16 files are automatically skipped, because it are binary files. EXAMPLES
Get input from stdin and write output to stdout. dos2unix dos2unix -l -c mac Convert and replace a.txt. Convert and replace b.txt. dos2unix a.txt b.txt dos2unix -o a.txt b.txt Convert and replace a.txt in ascii conversion mode. dos2unix a.txt Convert and replace a.txt in ascii conversion mode. Convert and replace b.txt in 7bit conversion mode. dos2unix a.txt -c 7bit b.txt dos2unix -c ascii a.txt -c 7bit b.txt Convert a.txt from Mac to Unix format. dos2unix -c mac a.txt mac2unix a.txt Convert a.txt from Unix to Mac format. unix2dos -c mac a.txt unix2mac a.txt Convert and replace a.txt while keeping original date stamp. dos2unix -k a.txt dos2unix -k -o a.txt Convert a.txt and write to e.txt. dos2unix -n a.txt e.txt Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt. dos2unix -k -n a.txt e.txt Convert and replace a.txt. Convert b.txt and write to e.txt. dos2unix a.txt -n b.txt e.txt dos2unix -o a.txt -n b.txt e.txt Convert c.txt and write to e.txt. Convert and replace a.txt. Convert and replace b.txt. Convert d.txt and write to f.txt. dos2unix -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt LOCALIZATION
LANG The primary language is selected with the environment variable LANG. The LANG variable consists out of several parts. The first part is in small letters the language code. The second is optional and is the country code in capital letters, preceded with an underscore. There is also an optional third part: character encoding, preceded with a dot. A few examples for POSIX standard type shells: export LANG=nl Dutch export LANG=nl_NL Dutch, The Netherlands export LANG=nl_BE Dutch, Belgium export LANG=es_ES Spanish, Spain export LANG=es_MX Spanish, Mexico export LANG=en_US.iso88591 English, USA, Latin-1 encoding export LANG=en_GB.UTF-8 English, UK, UTF-8 encoding For a complete list of language and country codes see the gettext manual: <http://www.gnu.org/software/gettext/manual/gettext.html#Language-Codes> On Unix systems you can use to command locale(1) to get locale specific information. LANGUAGE With the LANGUAGE environment variable you can specify a priority list of languages, separated by colons. Dos2unix gives preference to LANGUAGE over LANG. For instance, first Dutch and then German: "LANGUAGE=nl:de". You have to first enable localization, by setting LANG (or LC_ALL) to a value other than "C", before you can use a language priority list through the LANGUAGE variable. See also the gettext manual: <http://www.gnu.org/software/gettext/manual/gettext.html#The-LANGUAGE-variable> For Esperanto there is a special language file in x-method format. X-method can be used on systems that don't support Latin-3 or Unicode character encoding. Make LANGUAGE equal to "eo-x:eo". If you select a language which is not available you will get the standard English messages. DOS2UNIX_LOCALEDIR With the environment variable DOS2UNIX_LOCALEDIR the LOCALEDIR set during compilation can be overruled. LOCALEDIR is used to find the language files. The GNU default value is "/usr/local/share/locale". Option "-V" will display the LOCALEDIR that is used. Example (windows cmd): set DOS2UNIX_LOCALEDIR=c:/my_prefix/share/locale AUTHORS
Benjamin Lin - <blin@socs.uts.edu.au> Bernd Johannes Wuebben (mac2unix mode) - <wuebben@kde.org> Erwin Waterlander - <waterlan@xs4all.nl> Project page: <http://www.xs4all.nl/~waterlan/dos2unix.html> SourceForge page: <http://sourceforge.net/projects/dos2unix/> Freshmeat: <http://freshmeat.net/projects/dos2unix> SEE ALSO
iconv(1) dos2unix 2010-03-23 dos2unix(1)
All times are GMT -4. The time now is 04:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy