Line too long error Replace string with new line line character


 
Thread Tools Search this Thread
Operating Systems Solaris Line too long error Replace string with new line line character
# 1  
Old 05-16-2011
Line too long error Replace string with new line line character

I get a file which has all its content in a single row.
The file contains xml data containing 3000 records, but all in a single row, making it difficult for Unix to Process the file.

I decided to insert a new line character at all occurrences of a particular string in this file (say replacing "<record>" with "\n<record>", so that the file has multiple rows without impacting the xml data in it.

I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it.
I cannot 'fold' the file as it breaks the file at a fixed width, disrupting the xml tags and xml data.

How can I edit this file.
# 2  
Old 05-16-2011
Try GNU's version of sed. It seems to have no line length limit: sed, a stream editor. You can download GNU sed from sunfreeware.com.
# 3  
Old 05-17-2011
Quote:
Originally Posted by ducati
I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it..
Are you sure you tried a Perl script, the following seems to work (Tested on Solaris 5.10 sparc with Perl-5.8.4)
Code:
skrynesaver@busybox ~/tmp$ perl -w -Mstrict -e 'open my $long , ">" , "longLines.txt"; for (0..30000){print $long "<record><field1>data</field1><field2>other data </field2></record>";}'
skrynesaver@busybox ~/tmp$ wc  longLines.txt
      0   60003 1980066 longLines.txt
skrynesaver@busybox ~/tmp$ perl -w -Mstrict -e 'open(my $long, "<", "longLines.txt") || die "Could not open longLines.txt $!\n";my $data=readline($long);close $long;$data=~s/<record>/\n<record>/g;open my $short, ">", "shortLines";print $short $data;close $short'
skrynesaver@busybox ~/tmp$  wc  shortLines
  30001   90003 2010067 shortLines


Last edited by Skrynesaver; 05-17-2011 at 05:28 AM.. Reason: Clarified that I tested on a sparc box
This User Gave Thanks to Skrynesaver For This Post:
# 4  
Old 05-17-2011
Quote:
Originally Posted by ducati
I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it.
I cannot 'fold' the file as it breaks the file at a fixed width, disrupting the xml tags and xml data.
1) What is your file size ?
2) Is you sed/awk implementation version compiled for 64bit or 32bit plateform ?
(What gives the command file <yourfile>)
3) Which sed command did you try ?

Did you try something like :
Code:
cat infile | sed 's/</#</g' | tr '#' '\n'

(this UUOC is just for test purpose to see if sed can better handle it as a stream than as a file)

(or choose another character than the hash # , choose one so that it doesn't appear in your original file)

Also give a try to:
Code:
tr '<' '\n' <infile | sed '1!s/^/</'

# 5  
Old 05-17-2011
The perl script seems to be doing the trick. Thanks Skrynesaver
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to put a 80 character limit on a long topic line in markdown?

I have a topic line in markdown that spans more than 80 characters that i need to add a line break. Markdown is simply treating the line break as a brand new line instead of continuing as a topic line. Eg: # This is a very long line Markdown interprets it as This is a very long line (4 Replies)
Discussion started by: dragonpoint
4 Replies

2. Shell Programming and Scripting

Read line by line and replace string.

Hi, I currently have a problem that I need to read a file line by line. After I read it line by line there are some commands in which I have to change a specific string.(In my case, I have to make a script that changes all the passwords into hash value) Here is a sample input... (3 Replies)
Discussion started by: thebennnn
3 Replies

3. Shell Programming and Scripting

Replace and add line in file with line in another file based on matching string

Hi, I want to achieve something similar to what described in another post: The difference is I want to add the line if the pattern is not found. File 1: A123, valueA, valueB B234, valueA, valueB C345, valueA, valueB D456, valueA, valueB E567, valueA, valueB F678, valueA, valueB ... (11 Replies)
Discussion started by: jyu3
11 Replies

4. Shell Programming and Scripting

sed command to replace a line at a specific line number with some other line

my requirement is, consider a file output cat output blah sdjfhjkd jsdfhjksdh sdfs 23423 sdfsdf sdf"sdfsdf"sdfsdf"""""dsf hellow there this doesnt look good et cetc etc etcetera i want to replace a line of line number 4 ("this doesnt look good") with some other line ... (3 Replies)
Discussion started by: vivek d r
3 Replies

5. Shell Programming and Scripting

awk new line issue, saying string can't contain new line character

Hi , I am doing some enhancements in an existing shell script. There it used the awk command in a function as below : float_expr() { IFS=" " command eval 'awk " BEGIN { result = $* print result exit(result == 0) }"' } It calls the function float_expr to evaluate two values ,... (1 Reply)
Discussion started by: mady135
1 Replies

6. Emergency UNIX and Linux Support

Find a line using a condition and replace a string in that line

Hello, I have a 100 line code. I have given a sample of it below: ABC*654654*1*54.54*21.2*87*1*654654654654 CCC*FS*FS*SFD*DSF GGG*FGH*CGB*FBDFG*FGDG ABC*654654*1*57.84*45.4*88*2*6546546545 CCC*WSF*SG*FGH*GHJ ADA*AF*SFG*DFGH*FGH*FGTH I need to select the line starting with "ABC" its... (3 Replies)
Discussion started by: nithins007
3 Replies

7. Shell Programming and Scripting

Find a line using a condition and replace a string in that line

Hello, I have a 100 line code. I have given a sample of it below: ABC*654654*1*54.54*21.2*87*1*654654654654 CCC*FS*FS*SFD*DSF GGG*FGH*CGB*FBDFG*FGDG ABC*654654*1*57.84*45.4*88*2*6546546545 CCC*WSF*SG*FGH*GHJ ADA*AF*SFG*DFGH*FGH*FGTH I need to select the line starting with "ABC" its... (6 Replies)
Discussion started by: nithins007
6 Replies

8. Shell Programming and Scripting

error while replacing a string by new line character in sed

hi, when i am doing the following things getting error Can anyone please suggest i have a file where there is a line like the following branch=dev sdf dev jin kilii fin kale boyle dev james dev i want to search the existance of dev in the above line. cat "$file" | sed -n... (8 Replies)
Discussion started by: millan
8 Replies

9. Shell Programming and Scripting

search for a string ,replace the whole line with new line

hai i am very new to unix. i am having two files like this. first.properties cache.ZA.TL_CCY=SELECT trim(CCY_CODE)||trim(COUNTRY_CODE)||trim(CITY_CODE) AS... (4 Replies)
Discussion started by: kkraja
4 Replies
Login or Register to Ask a Question

Featured Tech Videos