Login or Register to Ask a Question and Join Our Community


Solaris

Line too long error Replace string with new line line character

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris Line too long error Replace string with new line line character
# 1  
Old 05-16-2011
Line too long error Replace string with new line line character
I get a file which has all its content in a single row.
The file contains xml data containing 3000 records, but all in a single row, making it difficult for Unix to Process the file.

I decided to insert a new line character at all occurrences of a particular string in this file (say replacing "<record>" with "\n<record>", so that the file has multiple rows without impacting the xml data in it.

I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it.
I cannot 'fold' the file as it breaks the file at a fixed width, disrupting the xml tags and xml data.

How can I edit this file.
# 2  
Old 05-16-2011
Try GNU's version of sed. It seems to have no line length limit: sed, a stream editor. You can download GNU sed from sunfreeware.com.
# 3  
Old 05-17-2011
Quote:
Originally Posted by ducati
I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it..
Are you sure you tried a Perl script, the following seems to work (Tested on Solaris 5.10 sparc with Perl-5.8.4)
Code:
skrynesaver@busybox ~/tmp$ perl -w -Mstrict -e 'open my $long , ">" , "longLines.txt"; for (0..30000){print $long "<record><field1>data</field1><field2>other data </field2></record>";}'
skrynesaver@busybox ~/tmp$ wc  longLines.txt
      0   60003 1980066 longLines.txt
skrynesaver@busybox ~/tmp$ perl -w -Mstrict -e 'open(my $long, "<", "longLines.txt") || die "Could not open longLines.txt $!\n";my $data=readline($long);close $long;$data=~s/<record>/\n<record>/g;open my $short, ">", "shortLines";print $short $data;close $short'
skrynesaver@busybox ~/tmp$  wc  shortLines
  30001   90003 2010067 shortLines

Last edited by Skrynesaver; 05-17-2011 at 04:28 AM.. Reason: Clarified that I tested on a sparc box
This User Gave Thanks to Skrynesaver For This Post:
ducati
# 4  
Old 05-17-2011
Quote:
Originally Posted by ducati
I have tried sed, awk and perl commands but probably these commands aren't processing the file with too long line in it.
I cannot 'fold' the file as it breaks the file at a fixed width, disrupting the xml tags and xml data.
1) What is your file size ?
2) Is you sed/awk implementation version compiled for 64bit or 32bit plateform ?
(What gives the command file <yourfile>)
3) Which sed command did you try ?

Did you try something like :
Code:
cat infile | sed 's/</#</g' | tr '#' '\n'

(this UUOC is just for test purpose to see if sed can better handle it as a stream than as a file)

(or choose another character than the hash # , choose one so that it doesn't appear in your original file)

Also give a try to:
Code:
tr '<' '\n' <infile | sed '1!s/^/</'

# 5  
Old 05-17-2011
The perl script seems to be doing the trick. Thanks Skrynesaver
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to put a 80 character limit on a long topic line in markdown?

I have a topic line in markdown that spans more than 80 characters that i need to add a line break. Markdown is simply treating the line break as a brand new line instead of continuing as a topic line. Eg: # This is a very long line Markdown interprets it as This is a very long line (4 Replies)
Discussion started by: dragonpoint
4 Replies

2. Shell Programming and Scripting

Read line by line and replace string.

Hi, I currently have a problem that I need to read a file line by line. After I read it line by line there are some commands in which I have to change a specific string.(In my case, I have to make a script that changes all the passwords into hash value) Here is a sample input... (3 Replies)
Discussion started by: thebennnn
3 Replies

3. Shell Programming and Scripting

Replace and add line in file with line in another file based on matching string

Hi, I want to achieve something similar to what described in another post: The difference is I want to add the line if the pattern is not found. File 1: A123, valueA, valueB B234, valueA, valueB C345, valueA, valueB D456, valueA, valueB E567, valueA, valueB F678, valueA, valueB ... (11 Replies)
Discussion started by: jyu3
11 Replies

4. Shell Programming and Scripting

sed command to replace a line at a specific line number with some other line

my requirement is, consider a file output cat output blah sdjfhjkd jsdfhjksdh sdfs 23423 sdfsdf sdf"sdfsdf"sdfsdf"""""dsf hellow there this doesnt look good et cetc etc etcetera i want to replace a line of line number 4 ("this doesnt look good") with some other line ... (3 Replies)
Discussion started by: vivek d r
3 Replies

5. Shell Programming and Scripting

awk new line issue, saying string can't contain new line character

Hi , I am doing some enhancements in an existing shell script. There it used the awk command in a function as below : float_expr() { IFS=" " command eval 'awk " BEGIN { result = $* print result exit(result == 0) }"' } It calls the function float_expr to evaluate two values ,... (1 Reply)
Discussion started by: mady135
1 Replies

6. Emergency UNIX and Linux Support

Find a line using a condition and replace a string in that line

Hello, I have a 100 line code. I have given a sample of it below: ABC*654654*1*54.54*21.2*87*1*654654654654 CCC*FS*FS*SFD*DSF GGG*FGH*CGB*FBDFG*FGDG ABC*654654*1*57.84*45.4*88*2*6546546545 CCC*WSF*SG*FGH*GHJ ADA*AF*SFG*DFGH*FGH*FGTH I need to select the line starting with "ABC" its... (3 Replies)
Discussion started by: nithins007
3 Replies

7. Shell Programming and Scripting

Find a line using a condition and replace a string in that line

Hello, I have a 100 line code. I have given a sample of it below: ABC*654654*1*54.54*21.2*87*1*654654654654 CCC*FS*FS*SFD*DSF GGG*FGH*CGB*FBDFG*FGDG ABC*654654*1*57.84*45.4*88*2*6546546545 CCC*WSF*SG*FGH*GHJ ADA*AF*SFG*DFGH*FGH*FGTH I need to select the line starting with "ABC" its... (6 Replies)
Discussion started by: nithins007
6 Replies

8. Shell Programming and Scripting

error while replacing a string by new line character in sed

hi, when i am doing the following things getting error Can anyone please suggest i have a file where there is a line like the following branch=dev sdf dev jin kilii fin kale boyle dev james dev i want to search the existance of dev in the above line. cat "$file" | sed -n... (8 Replies)
Discussion started by: millan
8 Replies

9. Shell Programming and Scripting

search for a string ,replace the whole line with new line

hai i am very new to unix. i am having two files like this. first.properties cache.ZA.TL_CCY=SELECT trim(CCY_CODE)||trim(COUNTRY_CODE)||trim(CITY_CODE) AS... (4 Replies)
Discussion started by: kkraja
4 Replies
Login or Register to Ask a Question