Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed: Working on a line Previous to a pattern.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed: Working on a line Previous to a pattern.
# 1  
Old 06-20-2010
Question Sed: Working on a line Previous to a pattern.
Hello everyone,

I am working with some train time tables, and i have hit a bit of a road block.

Using grep/sed i have done a reasonable job of parsing the html into comma delimited format, but NJ transit prints The Track number and status on a new line, and I would much prefer it all on a single line.

I more or less need to search for a line starting with a comma and then go to the previous line and remove the line break.

Let me show you where I am:

Code:
curl -silent http://dv.njtransit.com/mobile/tid-mobile.aspx?sid=NY |grep '<tr width="100%"' | sed 's/&nbsp;//g;s/<\/td>/,/g;s/<[^>]*>//g;s/  \/ /,/g;s/ to //g;/Track ,  ,/d;s/Track , /,/g;s/, /,/g'

Outputs....

Code:
7:00,A2257,Washington,AMTK,
,60 Min Late,
7:52,7673,Rahway,NEC,
Track 4,ALL ABOARD,
7:56,A165,Washington,AMTK,
,25 MINS LATE,
8:00,A2259,Washington,AMTK,
,15 MINS LATE,
8:02,7871,Trenton,NEC,
Track 7,BOARDING,
8:08,7273,Long Branch,NJCL,

I am a bit stumped and could really use a hand from some one with a bit more experience in such things.

Any advise would be greatly appreciated, Thanks guys!
# 2  
Old 06-21-2010
Hi

Please provide with the input file stream which your sed is receiving. It will help us to understand better.

Guru.
This User Gave Thanks to guruprasadpr For This Post:
mussen
# 3  
Old 06-21-2010
Sorry i should have made that more clear, In the first Code snip the first statement is :

Code:
curl -silent http://dv.njtransit.com/mobile/tid-mobile.aspx?sid=NY

So that is my input stream ^


So right now this outputs something like this for each train:
Code:
1:00,3205,Long Branch,NJCL,
Track 10, BOARDING,

I want to turn that into this

Code:
1:00,3205,Long Branch,NJCL,Track 10, BOARDING,

Last edited by mussen; 06-21-2010 at 01:55 AM.. Reason: oops
# 4  
Old 06-21-2010
Hi mussen
Not this. I mean, before sed, how does your input file look like. Similar to how you have provided the output you have got out of sed, provide us the input file to sed.

Guru.
# 5  
Old 06-21-2010
http://iamrobertely.com/toys/rail/forunix.com.text

Ahh, ok I just dumped that ^

Just the curl/and grep.

Perhaps i could just drop the return of every odd numbered line..... before i do any thing else....But this strikes me as too dirty, even for screen scraping.
# 6  
Old 06-21-2010
Hi
Does this help:

Code:
"your command currently" | sed 'N;s/\n/ /'

This will simply join every 2 lines.

Guru.
# 7  
Old 06-21-2010
Yes! Great, Ok Now i had tried that before but with little success, but Getting me to print the original string again made me realize the error of my ways.

The Trick is to run that first before the rest of the statement! Another gotcha is that NJ Transit is using windows Line breaks, so i have to dump those before I can get a reliable string. So i have to run a tr -d "\r" first, then Join every 2 lines. Wooo I really should have seen that, but i suppose i am running a tad low on sleep. Give me a moment to post the final command.

---------- Post updated at 02:33 AM ---------- Previous update was at 02:23 AM ----------

Code:
curl -silent http://dv.njtransit.com/mobile/tid-mobile.aspx?sid=NY |grep '<tr width="100%"' | tr -d "\r" | sed 'N;s/\n/ /;s/&nbsp;//g;s/<\/td>/,/g;s/<[^>]*>//g;s/  \/ /,/g;s/ to //g;s/, /,/g;s/ ,/,/g;s/,$//'

OK, thats what I have for my final statement, I just had to tidy it up a bit at the end, but its working great! Thanks for the help!

---------- Post updated at 02:48 AM ---------- Previous update was at 02:33 AM ----------

And here.....Is a single line that will Print Status of all the trains into a CSV called test.csv (even with a header)
Code:
echo "Time","Train","Destination","Line","Track","Status">test.csv && curl -silent http://dv.njtransit.com/mobile/tid-mobile.aspx?sid=NY |grep '<tr width="100%"' | tr -d "\r" | sed 'N;s/\n/ /;s/&nbsp;//g;s/<\/td>/,/g;s/<[^>]*>//g;s/  \/ /,/g;s/ to //g;s/, /,/g;s/ ,/,/g;s/,$/"/;s/,/","/g;s/^/"/;s/"Track"/""/g' >>test.csv

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete the previous line after pattern match?

Team, I am writing a shell script to perform few health checks of the system, where I need to delete the previous line in the text file after pattern match using sed (or) awk. Could you please help me out on this? For example, <td> <td style=color:green align=center> </td> </tr>... (6 Replies)
Discussion started by: Nagaraj R
6 Replies

2. Shell Programming and Scripting

How to print previous line of multiple pattern matched line?

Hello, I have below format log file, Comparing csv_converted_files/2201/9747.1012H67126.5077292103609547345.csv and csv_converted_files/22019/97447.1012H67126.5077292103609547345.csv Comparing csv_converted_files/2559/9447.1012H67126.5077292103609547345.csv and... (6 Replies)
Discussion started by: arvindshukla81
6 Replies

3. Shell Programming and Scripting

Sed: how to merge two lines moving matched pattern to end of previous line

hello everyone, im new here, and also programming with awk, sed and grep commands on linux. In my text i have many lines with this config: 1 1 4 3 1 1 2 5 2 2 1 1 1 3 1 2 1 3 1 1 1 2 2 2 5 2 4 1 3 2 1 1 4 1 2 1 1 1 3 2 1 1 5 4 1 3 1 1... (3 Replies)
Discussion started by: satir
3 Replies

4. Shell Programming and Scripting

sed : match one pattern then the next consecutive second pattern not working

Ive used this snippet of code on a solaris box thousands of times. But it isnt working on the new linux box sed -n '/interface LoopBack0/{N;/ ip address /p;}' *.conf its driving me nuts !! Is there something Im missing ? (7 Replies)
Discussion started by: popeye
7 Replies

5. Shell Programming and Scripting

awk to insert line previous to a pattern?

I have a very long line with certain patters embedded in there. I need to be able to read that line, and when it encounters that pattern, create a new line. I want the pattern to be the beginning of the new line. I thought sed or awk could do this, but everything I try in sed gives me a "sed... (2 Replies)
Discussion started by: Drenhead
2 Replies

6. Shell Programming and Scripting

Sed Comparing Parenthesized Values In Previous Line To Current Line

I am trying to delete lines in archived Apache httpd logs Each line has the pattern: <ip-address> - - <date-time> <document-request-URL> <http-response> <size-of-req'd-doc> <referring-document-URL> This pattern is shown in the example of 6 lines from the log in the code box below. These 6... (1 Reply)
Discussion started by: Proteomist
1 Replies

7. Shell Programming and Scripting

sed: how to move matched pattern to end of previous line

Hello, I'm new to this forum. I've been doing a lot of sed work lately and have found many useful tips on this forum. I've hit a roadblock in a project, though, and could really use some help. I have a text file with many lines like the following, i.e., some lines begin with a single word... (3 Replies)
Discussion started by: paroikoi
3 Replies

8. Shell Programming and Scripting

Append next line to previous line when one pattern not found

Hi, I need help for below scenario.I have a flat file which is having records seperated by delimiters which will represent each record for oracle table.My Control file will consider each line as one record for that table. Some of the lines are aligned in two/three lines so that records are... (4 Replies)
Discussion started by: kannansr621
4 Replies

9. Shell Programming and Scripting

Printing previous line based on pattern using sed

Hi, I have a written a shell script to get the previous line based on the pattern. For example if a file has below lines: ---------------------------------------------- #UNBLOCK_As _per #As per 205.162.42.92 #BLOCK_As_per #----------------------- #input checks abc.com... (5 Replies)
Discussion started by: Anjan1
5 Replies

10. UNIX for Dummies Questions & Answers

return previous line for pattern match

Hi, Need some idea on file processing, I have file like below, Processing al sources ... ...No value found : CHECK. Completed comparing all sources. Comparing schedulers... Processing al targets ... ...No value found : From above I need to extract the line where "No value... (4 Replies)
Discussion started by: braindrain
4 Replies
Login or Register to Ask a Question