Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove improperly placed newlines
Top Forums Shell Programming and Scripting Remove improperly placed newlines Post 302400642 by mikesimone on Wednesday 3rd of March 2010 07:27:29 PM
Old 03-03-2010
Remove improperly placed newlines
Hello, there. I have a file that's a horrible, horrible mess. (Basically, it's an export from a firewall config.) The people who generated the file didn't think that putting a newline in the middle of a hostname would ever be a problem. It is.

Here's an example of the stuff in the file:
Code:
Entity name: Random_Subnet
02F7B97A-9DA1-4393-A7E3-7F367356190C
Network address
192.168.151.0
Random_Subnet
Netmask
255.255.255.0
Entity name: AccessApplications_Group
E486A584-E708-4DB3-809E-F5685F2215E5
Access internet applications
Network Entity
Random_Subnet,Random_Subnet2,Random_
Subnet3,Random_Subnet4,Random_Subnet
5,Random_Subnet6

The part with the network address I have parsed, all well and good. But the second "entity name" entry, where there's a group, you can see that it puts line breaks in random places. There's no rhyme or reason - there doesn't have to be a special character or anything, it just puts that there.

This is the string I used to get it to that point:
Code:
tr -d '\t' < VPhosts.htm | sed 's/<.*>//g' | sed '/^$/d' | egrep -v "false|Symantec Gateway Security|Description|Locked|Read only|Caption|Sequence Revision|MAC address|UUID"

There's further string manipulation I do afterwards to get it into a proper comma-separated value file. But right here is the step where I need help.

I want to take every line which has commas in it, remove the line break from the end, and concatenate it with the line after. I want this only to happen on lines with commas, everything else should remain intact. I've tried everything in my limited skill set with sed, awk, perl and tr of which I could think, and I can't get it right. The closest I came was:

Code:
for i in `cat test.out`; do if [[ $i == , ]]; then echo -n $i; else echo $i; fi; done > test2.out

But that also put in a newline wherever there was a space, so it didn't help, either.

Can someone give me a little one-liner that will do this? It sounds so simple, but it has gone way beyond my meager ability at scripting.
 

10 More Discussions You Might Find Interesting

1. Solaris

Improperly formatted value for 'tftp-server' error while booting the target from LAN

Hi, I wish to install solaris 10 on a target machine (t1000) by using Jumpstart. I have configured by jumpstart environment for the same. When i boot the target with the option "boot net -v install" i get the following error... {0} ok boot net -v install Boot device: /pci@7c0/pci@0/network@4... (3 Replies)
Discussion started by: hemalsid
3 Replies

2. Shell Programming and Scripting

Need to remove improperly formatted fortran output line from files, tried sed

I have been trying to remove some improperly formatted lines of output from fortran code I have been using. The problem is that I have some singularities in the math for some points that causes an incorrectly large value to be reported that exceeds the normal formating set in the code resulting in... (2 Replies)
Discussion started by: gillesc_mac
2 Replies

3. Shell Programming and Scripting

Newlines in shell variables

Hello, I'm trying to create a shell variable with newlines inside it, so that when I echo the variable and pipe it to, say, awk, it output with the newlines. Why is this so problematic? I frankly don't know, but BASH seems to be stripping my variable of newlines. Here's an example $ cat... (5 Replies)
Discussion started by: narcvs
5 Replies

4. Shell Programming and Scripting

awk - need to remove unwanted newlines on match

Context: I need to remove unwanted newlines from a data file listing books and associated data. Here is a sample listing ( line numbers included ): 1 360762| Skip-beat! 14 /| 9781421517544| nb | 2008.| Nakamura, Yoshiki.| NAKAMUR | Kyoko Mogami followed 2 her true love Sho to Tokyo to... (6 Replies)
Discussion started by: Bubnoff
6 Replies

5. Shell Programming and Scripting

Transpose with two newlines as delimiter

Hi Guys, I have data in a file as follows: a 1 2 3 b 4 5 6 a 6 7 8 a 4 7 9 b 6 8 5 c 0 8 7 So the number of rows which have data is variable (2 for the first group, one for the second group and three for the third group), but the delimiters between the... (10 Replies)
Discussion started by: npatwardhan
10 Replies

6. Shell Programming and Scripting

sed remove newlines and spaces

Hi all, i am getting count from oracle 11g by spooling it to a file. Now there are some newline characters and blank spaces i need to remove these. pl provide me a awk/sed solution. the spooled file is attached. i tried this.. but not getting req o/p (6 Replies)
Discussion started by: rishav
6 Replies

7. Shell Programming and Scripting

removing newlines after a certain word.

Hello! This is my first post here. I have a file with text like: A</title> B C</title> D I need to format it to: AB CD I am trying to use sed: sed 's/<//title>\n/ /g' file > newfile to delete </title> and the newline character, but the file is unchanged because there are... (3 Replies)
Discussion started by: DaytonCPS
3 Replies

8. UNIX for Dummies Questions & Answers

Remove newlines

Hi buddy's my file are like this: s.no,name,band,sal 1,"suneel",,10 2,"bargav sand",,20 30," ebdug gil",,4 but i want s.no,name,band,sal 1,"suneel",,10 2,"bargav sand",,20 30,"ebdug gil",,4 any command or Shell script for this. please help me it's urgent to implement (33 Replies)
Discussion started by: Suneelbabu.etl
33 Replies

9. UNIX for Dummies Questions & Answers

Using find with awk to remove newlines

I want to list all html files present in a directory tree, the remove the newline and get one string with a space between files find /home/chrisd/Desktop/seg/geohtml/ -name '*.html' | awk BEGIN{FS=\r} '{print}' ---------- Post updated at 06:47 PM ---------- Previous update was at 06:25 PM... (5 Replies)
Discussion started by: kristinu
5 Replies

10. UNIX for Beginners Questions & Answers

Remove newlines and carriage return from a csv file using UNIX

I need to remove new lines and carriage returns from csv file. Is there anything other than sed and gwak by which we could achieve this ? Any suggestions ? (3 Replies)
Discussion started by: A_Gaddale
3 Replies

LEARN ABOUT MINIX

securenets

securenets(4)							   File Formats 						     securenets(4)

NAME

       securenets - configuration file for NIS security

SYNOPSIS

       /var/yp/securenets

DESCRIPTION

       The   /var/yp/securenets  file  defines	the  networks  or hosts which are allowed access to information by the Network Information Service
       ("NIS").

       The format of the file is as follows:

	 o  Lines beginning with the ``#'' character are treated as comments.

	 o  Otherwise, each line contains two fields separated by white space.	The first field is a netmask, the second a network.

	 o  The netmask field may be either 255.255.255.255 (IPv4), ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff (IPv6) , or the string ``host''  indi-
	    cating that the second field is a specific host to be allowed access.

       Both  ypserv(1M)  and  ypxfrd(1M)  use the /var/yp/securenets file.  The file is read when the  ypserv(1M) and ypxfrd(1M) daemons begin. If
       /var/yp/securenets is present, ypserv(1M) and ypxfrd(1M) respond only to IP addresses in the range given.  In order for	a  change  in  the
       /var/yp/securenets file to take effect, you must kill and restart any active daemons using ypstop(1M) and ypstart(1M).

       An  important  thing  to note for all the examples below is that the server must be allowed to access itself. You accomplish this either by
       the server being part of a subnet that is allowed to access the server, or by adding an individual entry, as the following:

       hosts 127.0.0.1

EXAMPLES

       Example 1: Access for Individual Entries

       If individual machines are to be give access, the entry could be:

       255.255.255.255	   192.9.1.20
       or

       host 192.0.1.20

       Example 2: Access for a Class C Network

       If access is to be given to an entire class C network, the entry could be:

       255.255.255.0  192.9.1.0

       Example 3: Access for a Class B Network

       The entry for access to a class B network could be:

       255.255.0.0    9.9.0.0

       Example 4: Access for an Invidual IPv6 Address

       Similarly, to allow access for an individual IPv6 address:

       ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff	fec0::111:abba:ace0:fba5e:1
       or

       host  fec0::111:abba:ace0:fba5e:1

       Example 5: Access for all IPv6 Addresses Starting with fe80

       To allow access for all IPv6 addresses starting with fe80:

       ffff::  fe80::

FILES

       /var/yp/securenets      Configuration file for
				NIS security.

SEE ALSO

       ypserv(1M), ypstart(1M), ypstop(1M), ypxfrd(1M)

NOTES

       The Network Information Service (NIS) was formerly known as Sun Yellow Pages (YP).  The functionality of the two remains the same; only the
       name  has  changed. The name Yellow Pages is a registered trademark in the United Kingdom of British Telecommunications plc, and may not be
       used without permission.

SunOS 5.10							    26 Apr 1999 						     securenets(4)
All times are GMT -4. The time now is 02:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy