Using sed to "unfold" a file...


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using sed to "unfold" a file...
# 1  
Old 04-04-2009
Power Using sed to "unfold" a file...

I have a file that has fields with text "folded" over more than one line. I'd like to parse the file for information, however I need to "unfold" it first.

My gut tells me that sed is the command to use, however trying to make it work modifying examples I have found here is just not working. And I just know it must be the simplest thing in the world!

How do I unfold my file using sed???

A folded line is a line with a field of information that has been purposely broken across multiple lines. A CRLF is followed by a single space on the next line.

Anyone up for showing a newbie how green he is?

EDIT: More Information
---
The desired output would be when sed takes any line that begins with a single whitespace and combines it with the previous line (ie unfolding it). Below I have provided two examples.

"Folded" line:
---
Code:
DESCRIPTION:Lorem ipsum dolor sit amet\, consectetuer adipiscing elit\, sed
  diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat v
 olutpat. Ut wisi enim ad minim veniam\, quis nostrud exerci tation ullamcor
 per suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem v
 el eum iriure dolor in hendrerit in vulputate velit esse molestie consequat
 \, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et 
 iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis
  dolore te feugait nulla facilisi.

"Unfolded" line:
---
DESCRIPTION:Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.

Too Much Information:
---
As more of a learning project, I am writing a script that looks at my gCal calendar and creates a time sheet of my activities. I am downloading the calendar on demand from Google using curl. I am also downloading from my own private website a client table, and a job table. Using awk and grep I am parsing information from all three files and creating text files (work reports, invoices, etc.) I "think" I already have the code to identify events, allocate ISO week numbers, extrapolate event data, do date/duration calculations, do cost calculations and output the results to a text file. A useful application that enables me to learn more about scripting and saves me time in the process.

I would like to be able to parse the DESCRIPTION field for information I might add to the event while I am out on my iPhone (job code, client code, rate code, materials, etc.) and have the final output adjusted accordingly. The problem is that in an ics file any field over 75 octets is "folded". "That is, a long line can be split between any two characters by inserting a CRLF immediately followed by a single linear white space character (i.e., SPACE, US-ASCII decimal 32 or HTAB, US-ASCII decimal 9). Any sequence of CRLF followed immediately by a single linear white space character is ignored (i.e., removed) when processing the content type." That's from From RFC2445.

The first goal is to have a csv file that I can stick into Excel, Word, Pages. There will be an immediate time saving from manually creating time sheets. I am almost there already. The second goal is to use groff to automatically create a PDF with a logo. If the results are accurate, I could just have it automatically email them to clients once a week. The third goal is to use cgi scripting and Apache to create a simple web page that will let me generate a work report or an invoice based on some basic information that a user provides (date range, client, etc.) I could then create invoices on the fly or even have clients generate their own work reports. Maybe have another page that allows me to edit the client and job tables... But now I am getting waaaaaaaay ahead of myself.

Last edited by Franklin52; 04-05-2009 at 09:24 AM..
# 2  
Old 04-04-2009
Post an example and desired output please. Or have a look into sed howtos/guides describing multiline pattern options like "N" for example.
# 3  
Old 04-04-2009
More Information...

[Deleted copy of original post...]

Last edited by melmaninga; 04-05-2009 at 04:36 PM..
# 4  
Old 04-04-2009
Sorry...

Quote:
Originally Posted by zaxxon
Post an example and desired output please. Or have a look into sed howtos/guides describing multiline pattern options like "N" for example.
Sorry. More info provided in original post.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

3. Post Here to Contact Site Administrators and Moderators

Suggestion: adding two new groups "sed" and "awk"

Majority of the questions are pertaining file/string parsing w.r.t sed or awk It would be nice to have these two as their own sub category under shell-programming-scripting which can avoid lot of duplicate posts. (1 Reply)
Discussion started by: jville
1 Replies

4. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

5. Shell Programming and Scripting

Using sed to find text between a "string " and character ","

Hello everyone Sorry I have to add another sed question. I am searching a log file and need only the first 2 occurances of text which comes after (note the space) "string " and before a ",". I have tried sed -n 's/.*string \(*\),.*/\1/p' filewith some, but limited success. This gives out all... (10 Replies)
Discussion started by: haggismn
10 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. Shell Programming and Scripting

Simplify Bash Script Using "sed" Or "awk"

Input file: 2 aux003.net3.com error12 6 awn0117.net1.com error13 84 aux008 error14 29 aux001.ha.ux.isd.com error12 209 aux002.vm.ux.isd.com error34 21 alx0027.vm.net2.com error12 227 dux001.net5.com error123 22 us008.dot.net2.com error121 13 us009.net2.com error129Expected Output: 2... (4 Replies)
Discussion started by: sQew
4 Replies

8. Shell Programming and Scripting

Help to change the file with "sed" and "awk"

Hi experts I want your help to change the file format to my wanted version, please give me a hand thanks $cat file install pass make os pass make build kernel failed usb storage pass chane to | *install* | *make os* | *make build kernel* | *usb storage* | | pass | pass... (7 Replies)
Discussion started by: yanglei_fage
7 Replies

9. Shell Programming and Scripting

cat $como_file | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g'

hi All, cat file_name | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g' Can this be done by using sed or awk alone (4 Replies)
Discussion started by: harshakusam
4 Replies

10. Shell Programming and Scripting

"sed" to check file size & echo " " to destination file

Hi, I've modified the syslogd source to include a thread that will keep track of a timer(or a timer thread). My intention is to check the file size of /var/log/messages in every one minute & if the size is more than 128KB, do a echo " " > /var/log/messages, so that the file size will be set... (7 Replies)
Discussion started by: jockey007
7 Replies
Login or Register to Ask a Question