Sponsored Content
Top Forums Shell Programming and Scripting Breaking long lines into (characters, newline, space) groups Post 302316337 by cfajohnson on Thursday 14th of May 2009 08:21:15 PM
Old 05-14-2009
Quote:
Originally Posted by rowie718
I came up with this script. It seems hackish and very inefficient, but it works. I would love for someone to help me come up with a better way since this script takes almost 10 full minutes to parse a text file into less than 7000 lines.
[/code]

For a file that size, you should use awk.
Quote:
[code]
Code:
#!/bin/ksh

echo "where is the ldif file located that you would like to parse?"
read response
ldiffile=$response


Why not simply:

Code:
read ldiffile

Quote:
Code:
while read line
do

x=`echo $line | wc -c`


You don't need an external command to get the length of a variable's contents:

Code:
x=${#line}

Quote:
Code:
while [ $x -gt 79 ]
do

sed 's/./\
 /79' $ldiffile > /test.ldif
mv /test.ldif $ldiffile
x=$x-79

done

done < $ldiffile


Code:
awk 'length > 79 { while ( length($0) > 79 ) {
    printf "%s\n ", substr($0,1,79)
    $0 = substr($0,80)
  }
  if (length) print
  next
}
{print}' "$FILE"

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove trailing newline characters

Hello , I have the folowing scenario : I have a text file as follows : (say name.txt) ABC DEF XYZ And I have one more xml file as follows : (say somexml.xml) <Name>ABC</Name> <Age>12</Age> <Class>D</Class> <Name>XYZ</Name> <Age>12</Age> <Class>D</Class> <Name>DEF</Name>... (7 Replies)
Discussion started by: shweta_d
7 Replies

2. UNIX for Dummies Questions & Answers

non-breaking space question

Might anyone know how to make a nbsp (160|0xA0) character? I am using a Dell Latitude D620 running Windows XP and then starting Exceed 9.0 defaulting to native window emulation for my X (us.kbf keymapping) (Latin-1 symbol set I believe) and calling an xterm (fontdefault, whatever that might be)... (1 Reply)
Discussion started by: runmeat6
1 Replies

3. Shell Programming and Scripting

Replace long space to become one space?

Hi, i have the log attached. Actually i want the long space just become 1 space left like this : Rgds, (12 Replies)
Discussion started by: justbow
12 Replies

4. Ubuntu

Disk Space lost mysteriously upon breaking a process.

Hi All, Today when I was working on a script to generate custom wordlist. So I ran a script and the output was directed to /tmp. The disk space was around 19 gb. While the script was running, I decided to direct the o/p file to my 1TB drive. So I broke the run using Ctrl + C. Now when I... (4 Replies)
Discussion started by: morningSunshine
4 Replies

5. UNIX for Dummies Questions & Answers

Breaking up a text file into lines

Hi, I have a space delimited text file that looks like the following: BUD31 YRI 2e-06:CXorf15 YRI 3e-06:CREB1 YRI 4e-06 FLJ21438 CEU 3e-07:ETS1 CEU 8e-07:FGD3 CEU 2e-06 I want to modify the text file so that everytime there is a ":", a new line is introduced so that the document looks... (3 Replies)
Discussion started by: evelibertine
3 Replies

6. Shell Programming and Scripting

cutting long text by special char around 100 byte and newline

Regard, How can i cut the text by special char(|) around 100 byte and write the other of the text at newline using Perl. ... (3 Replies)
Discussion started by: Shawn, Lee
3 Replies

7. Shell Programming and Scripting

awk: searching for non-breaking-space

This code shal search for the non-breaking space 0xA0 though it returns the error "fatal: attempt to use scalar 'nbs' as array" Can somebody help? awk --non-decimal-data -v nbs="0xA0" '{if($0 in nbs) {print FILENAME, NR}}' *.txt (1 Reply)
Discussion started by: sdf
1 Replies

8. Shell Programming and Scripting

Breaking lines which contains more than 50 characters in a file

Hi, I have a file which contains many lines. Some of them are longer than 50 chars. I want to break those lines but I don't want to break words, e.g. the file This is an exemplary text which should be broken aaaaaa bbbbb ccccc This is the second line This line should also be broken... (3 Replies)
Discussion started by: wenclu
3 Replies

9. Shell Programming and Scripting

Newline characters in fields of a file

My source file is pipe delimeted file with 53 fields.In 33 rd column i am getting mutlple new line characters,dule to that record is breaking into multiple records. Note : here record delimter also \n sample Source file with 6 fields : 1234|abc| \nabcd \n bvd \n cde \n |678|890|900\n ... (6 Replies)
Discussion started by: lakshmi001
6 Replies

10. Shell Programming and Scripting

Mailx appending exclamation mark and newline in a long line

Hi, I have a shell script which automates reporting and at times, requires the report line to be very long (sometimes as long as 2131 chars). The output I get is similar to this: XXXX XXXXXXX 16:15 3.24% 5.07% 3.69% 5.23% 3.68% 4.06% 3.57% 5.03% 4.31% 5.11% 3.49% 4.19% 4.31% ... (2 Replies)
Discussion started by: gilberteu
2 Replies
SHELL-QUOTE(1p) 					User Contributed Perl Documentation					   SHELL-QUOTE(1p)

NAME
shell-quote - quote arguments for safe use, unmodified in a shell command SYNOPSIS
shell-quote [switch]... arg... DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands or files with embedded white space or shell globbing characters safely. Here are a few examples. EXAMPLES
ssh preserving args When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and passes them to "$SHELL -c". This doesn't work as intended: ssh host touch 'hi there' # fails It creates 2 files, hi and there. Instead, do this: cmd=`shell-quote touch 'hi there'` ssh host "$cmd" This gives you just 1 file, hi there. process find output It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote: eval set -- `find -type f -print0 | xargs -0 shell-quote --` debug shell scripts shell-quote is better than echo for debugging shell scripts. debug() { [ -z "$debug" ] || shell-quote "debug:" "$@" } With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can. save a command for later shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are things the user can't pass through), you can do something like this: user_switches= while [ $# != 0 ] do case x$1 in x--pass-through) [ $# -gt 1 ] || die "need an argument for $1" user_switches="$user_switches "`shell-quote -- "$2"` shift;; # process other switches esac shift done # later eval "shell-quote some-command $user_switches my args" OPTIONS
--debug Turn debugging on. --help Show the usage message and die. --version Show the version number and exit. AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions. AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.8.4 2005-05-03 SHELL-QUOTE(1p)
All times are GMT -4. The time now is 10:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy