Splitting a line


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Splitting a line
# 15  
Old 07-05-2007
Yeah, this really isn't working. This script will work with the files which contain the string

======Document
2 of 8

However, I'm now seeing that many documents I'm getting contain the string
=======Document 2 of 8

And the vast majority contain the string
==========
Document 2 of 8

I have a really nice script that will separate out all the text I need and dump it into independent text files based on the third string above.

The purpose of this exercise has been to get all the text files I'm getting formatted to appear the same way so that I could uniformly apply the splitting script.

You can see my previous efforts on that front here
https://www.unix.com/shell-programmin...-renaming.html

I don't think your script above will help with the files with the string
=======Document 2 of 8

Simon
P.S. I'll be offline for the rest of the night. If anybody has any suggestions, I'll come back to this tomorrow. Although I'm ready to dump the entire project and just do it manually. Thank you for all your help thus far!
# 16  
Old 07-09-2007
Hi all,
I'm still having problems here and am wondering if anyone can help. I've made progress on some of my other issues and the only issue yet to resolve is splitting lines in a series of .txt files of the form "==================Document 1 of 11"
into the following
"===================
Document 1 of 11"

There are hundreds of .txt files. I'm using sed. I've tried using Dr. L's scripts above but can't for the life of me get them to work. I've tried just invoking sed at the command line and inserting newline characters; I've tried scripts with the 'N' command; nothing works.

The original files are divided into sections such as this:

==============================================================================Document 5 of 6 (Note, in the file, these digits are all on one line)












Tories under fire for taking cash from bailed-out firms; OPENING THEIR
WALLETS:[FINAL Edition]
LARRY JOHNSRUDE, Legislature Bureau Chief. Edmonton Journal. Edmonton,
Alta.:Apr 2, 1997. p. A.3

Author(s): LARRY JOHNSRUDE, Legislature Bureau Chief

Dateline: Edmonton

Section: CANADA

Publication title: Edmonton Journal. Edmonton, Alta.: Apr 2, 1997. pg. A.3

Source type: Newspaper

ProQuest document 21512919
ID:

Text Word Count 574

Document URL: http://proquest.umi.com/
pqdweb?did=21512919&Fmt=3&clientId=14119&RQT=309&VName=PQD

Abstract (Document Summary)

Alberta Conservative party officials are checking whether, in taking money from
the province's former waste treatment arm, they broke their own policy against
accepting donations from government corporations.

Alberta Special Waste Management System contributed $2,400 to Conservative
party coffers early in 1996 in the midst of a takeover by Bovar Inc., the
government's former partner in the Swan Hills plant.

Party financial records filed with Elections Alberta indicate the donation was
made Jan. 10, 1996, while the government still had a stake in the company.



Full Text (574 words)

(Copyright The Edmonton Journal)

Alberta Conservative party officials are checking whether, in taking money from
the province's former waste treatment arm, they broke their own policy against
accepting donations from government corporations.

Alberta Special Waste Management System contributed $2,400 to Conservative
party coffers early in 1996 in the midst of a takeover by Bovar Inc., the
government's former partner in the Swan Hills plant.

Party financial records filed with Elections Alberta indicate the donation was
made Jan. 10, 1996, while the government still had a stake in the company.

The province announced in July 1995 that it was paying Bovar Inc. $147.5
million to take its 40-per-cent interest in the Swan Hills plant off its hands.

The deal wasn't finalized until July 1996.

Peter Elzinga, executive director of the provincial Tories, said Tuesday he
will check on whether accepting the donation was contrary to party policy.

The Liberals say the donation was unethical.

``Even if there aren't any laws against this, you would think the government
would refuse to take money from anyone that they're negotiating with,'' said
Liberal environment critic Debby Carlson.

``It's absolutely not right. You're paying yourself with taxpayers' money.''

Bovar president Monty Davis said he saw nothing wrong with the donation, which
went to purchase a table at Premier Ralph Klein's fund-raising dinner in
Calgary.

``We wanted to hear what was being said at the dinner,'' Davis said. ``We
though it important to understand the political direction of the province.''

He said the government had only a non-financial interest in Alberta Special
Waste at the time because terms of Bovar's takeover had already been set.

The provincial Elections Act prohibits a political party from accepting
donations from government-controlled companies.

But Bill Sage, director of financial operations for Elections Alberta, said the
list of prohibited corporations didn't include Alberta Special Waste. He
suggested it may not have been on the list because the government didn't have a
controlling interest in it.

Bovar contributed $2,000 to the Conservative party last year.

Alberta Special Waste Management wasn't the only troubled company to give the
Conservatives a campaign contribution.

Cash-strapped Canadian Airlines International, which received a government-
backed loan to stay in business, contributed $4,200 to party coffers. Eaton's
Canada, currently seeking bankruptcy protection, gave $4,000.

In all, the Tories received $2.8 million in donations in 1996, which gave them
a $3-million surplus going into the March 11 election campaign.

The party had budgeted $1.7 million for the campaign. It isn't required to file
election expense statements until June.

By comparison, the Liberals received $644,000 in contributions in 1996, which
left them with a $9,900 deficit.

A Liberal party official called the deficit an accounting procedure rather than
a financial shortfall.

OPENING THEIR WALLETS

Some companies that donated to the Alberta Progressive Conservatives in 1996:

Alberta Special Waste Management Systems -- $2,400

Alberta Power Ltd. - $4,200

Amoco Canada Petroleum Co -- $13,900

[Table]

Baton Broadcasting Inc. -- $3,000

CFRN Television -- $5,000

CHED -- $800

Calgary Sun -- $600

Edmonton Sun -- $720



Canadian Airlines -- $4,200

Dow Chemical Canada Inc. -- $6,100

[Table]

Earl's Restaurants -- $10,000

Eaton's Canada -- $4,000



Edmonton Power -- $6,875

Husky Oil -- $7,500

[Table]

Imperial Oil -- $10,000

Nova Corp -- $14,400



Some companies that donated to the Alberta Liberals:

Amoco Canada

Petroleum Co. -- $7,300

[Table]

Calgary Sun -- $450

Edmonton Sun -- $500



Canwest Global Communications -- $5,000

[Table]

CFRN Television -- $450

Canadian Airlines -- $450



*** Infomart-Online ***

Credit: THE EDMONTON JOURNAL
=========================Document 6 of 6

I'm a definite newbie, but have learned a little bit through the course of the project. Simplicity is a virtue for me, as are explanations of what the various components of any script does.

I'm grateful for your time.
# 17  
Old 07-10-2007
the below script should work for you. I tested with the supplied file from your last post:

Note:
A. needs to be all on one line, or use a \

#!/usr/bin/ksh
( awk '$1!~/==Document/{print $0 }' $1; awk '$1~/==Document/{print substr($1,1,25) "\n" "Document" $NF }' $1)

-----------------
B. I used $1 as input, but you could easily put this inside a for loop instead,"but keep the $1's in the awk! Smilie
and substitute a variable for the $1 .
C. of course, redirect to a file with > if you want to save the output.

example:
#!/usr/bin/ksh
for INPUTT in `ls`
do

( awk '$1!~/==Document/{print $0 }' $INPUTT; awk '$1~/==Document/{print substr($1,1,25) "\n" "Document" $NF }' $INPUTT ) > ${INPUTT}_out

done
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a line in two variables

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 acl,user_xattr 1 1I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 )I want PART1=>>>>>UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 ... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

Splitting one line into multiple lines

Hi, I have one problem scenorio as below. my source file is : cat input_file. "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","y ou." my output should be like. "hi","there","how","are","you?", "It","was","great","working","with",... (7 Replies)
Discussion started by: abhilash_nakka
7 Replies

3. Shell Programming and Scripting

Splitting Single line into multiple line

Hi All, I am reading a line from a file and writing it to other file. Whenever I got a particular line then I want that line to be splited into 4 line and written it to new file. e.g My line is U_ABC connector3 pin24E connector4 pin25E connector5 pin26E connector6 pin27E connector7... (2 Replies)
Discussion started by: diehard
2 Replies

4. Shell Programming and Scripting

Splitting file based on line numbers

Hello friends, Is there any way to split file from n to n+6 into 1 file and (n+7) to (n+16) into other file etc. f.e I have source pipe delimated file with 20 lines and i need to split 1-6 in file1 and 7-16 in file2 and 17-20 in file 3 I need to split into fixed number of file like 4 files... (2 Replies)
Discussion started by: Rizzu155
2 Replies

5. Shell Programming and Scripting

Splitting a line with pattern in Sed - Unix

I have a file with single line, that line contains just like the following sample 00200100293^30^1^bla bla ...._______To: zabell00200100293^30^3^aSub00200100293^30^4^ellaCc: Sanders,De on my desk__________00200100293^30^4^___________________________________00A00ABC0293^30^1^something___To: some... (10 Replies)
Discussion started by: Vasan
10 Replies

6. Shell Programming and Scripting

Splitting 12M line file

I have a 12M line file that I need to split into smaller files but I'm getting a csplit: virtual memory exhausted error How can I quickly split this file? This is sze -r-xr-xr-x+ 1 VERGE Domain Users 1158288000 May 4 16:31 inputfile This is the command I've tried... csplit... (4 Replies)
Discussion started by: verge
4 Replies

7. Shell Programming and Scripting

Need help in splitting a line into fields in shell scripting

I have a line of more than 3000 bytes which will contain & as fields separator..I am using following awk command ..Its working but its not accepting the line more than 3000 bytes...Anyother alternate solution even in othe shell command also fine... awk -F '&' '{for( i=1; i<=NF; i++ ) print $i}'... (2 Replies)
Discussion started by: punithavel
2 Replies

8. Shell Programming and Scripting

Splitting the line in multiple lines

Hi Guys, I need a help. I am pretty new to the shell level programing. I was trying to split a long lines to a multiple lines with few conditions. I goggle for the code and found some snippets and tried to modified it but I got few strange problems too. I have to split the lines if line is ... (3 Replies)
Discussion started by: dd_sh
3 Replies

9. UNIX for Dummies Questions & Answers

Line Splitting

Hi, I want to make a new line between for eg: 0102030405 to make it look like: 01 02 03 04 05 by using sed commands. I'm just wondering how would I do this? p.s i'm new to this unix programming. (7 Replies)
Discussion started by: evoGage
7 Replies

10. UNIX for Dummies Questions & Answers

Splitting a line up

9600012301F TF02FT T03FFTF04TF 05T FF06TTTT I have to split this line up into this 96000123 01F TF 02FT T 03FFTF 04TF 05T FF 06TTTT Can anyone please Help?? (1 Reply)
Discussion started by: lilas
1 Replies
Login or Register to Ask a Question