OS X (Apple)

Hey RubiC!

Thanks a lot for your followup! I'm sorry but I think that you're too advanced for me...
I have no idea on how to combine bash commands with HTML and where to insert the new code into my script.

For now I have that:

Code:

curl -s "https://www.dotmed.com/equipment/2/92/1209/all/offset/0/all?key=&limit=20&price_sort=descending&cond=all&continent_filter=0&zip=&distance=5&att_1=0&att_row_num=1&additionalkeywords=&country=ES" | 
awk '
/href.*view more/ {sub (/^[^<]*<a href="/, "curl -s https://www.dotmed.com")
                         sub (/">.*$/, "")
                         print} ' | 
sh | 
awk ' match ($0, "id=\"(" IDS ")\"")  ||
 /<\/*title>/    {gsub (/<[^>]*>/, _)
                  print >> "/Users/MyUser/Desktop/test.txt"
                 } ' IDS="price|condition|date_updated"

I tried already to replace the portion:

Code:

/<\/*title>/    {gsub (/<[^>]*>/, _)
                  print >> "/Users/MyUser/Desktop/test.txt"
                 } ' IDS="price|condition|date_updated"

with your new code:

Code:

/^<!DOCTYPE/    {printf RS
                 }
END             {printf RS
                 } ' IDS="price|condition|date_updated|in_stock" ORS=","

But it yielded a blanc "page" on my terminal. Plus I'm not sure at all on how to export to a file?
Again, I'm truly sorry to be such a burden and would totally understand if you weren't able to help me further!

Oh! And one last thing: Having a coma is perfect, I'll try to deal with the "iner" comas afterwards. Using Excel, I can still fine tune that pretty easily I guess...

Thanks anyways and all the best,

Ardzii

Well, I said "add", not "replace". Add the lines after the print statement.

EDIT: And, yes, replace this line:

Code:

                 } ' IDS="price|condition|date_updated"

It worked like a charm RubiC!
I mean, I'm certain you expected it to work... :P

I even replaced the "," by a ";" which help me import the txt or csv to excel!
Now I'll be working on building something to have the looooonnnngggg line converted into columns!

Thanks a lot for your patience and your kind help!

Best!

Ardzii

EDIT: Oh yes, one quick word! I'm directly taking a Bash course on Udemy... hopefully I'll come back stronger next time!

What "looooonnnngggg line"? There should be admittedly somewhat lengthy lines with a title and max. four more fields if you specify four IDS (N fields for N IDS).

EDIT: But - hold on - I see you redirecting print >> "/Users/MyUser/Desktop/test.txt"? If so, you need to redirect the printf RS as well! Or, redirect the entire output of the pipe.

Another comment.
"href.*view more" works the same with ERE and RE, so egrep can be replaced by grep.
And because sed takes an RE and does not yet have one, you can move it to sed

Code:

link=$(
  curl "your-link-here" |
  sed -n '/href.*view more/ s/.*href="\([^"]*\).*/\1/p'
)

If lines break at certain logical points one does not need \ at the end.
--
Just seeing bakunin got the sed trick, too. Maybe my explanations add some value.

Hey Bakunin! Hi MadeinGermany!

Thanks a lot you guys for your interventions!

@Bakunin:
I knew that declaring the variable so that it can be used elsewhere. However, in the course I'm taking they didn't extend on the value added of it. But I will look KornShell up now thanks to your intervention!
I see that you used local (which I had not seen yet) rather than declare (this was the one I knew about): Any particular reason? Regarding the "-i" I guess it stands for integer?

I loved your comment regarding indexes! You're right the clearer the better! Same for the comment... I'll give your option a try and see how it works for me.

@Bakunin@MadeinGermany:
About sed|grep|awk you must be both right! To tell you the truth, I haven't got to the grep and sed sections yet. I tried some grep from lines I found on the web and as for the sed: I literally copied and pasted it to my script from the web (someone was trying to get rid of the same thing).

Let me keep on going with my Bash course and I'll come back with an update better version.

Thanks again to you both and have a good one!

EDIT: Just tried with the few changes you suggested and for some reason my terminal returns "local: can only be used in a function" and there is definitely no function in my script ... I will try using declare, what do you think?

There is more about declaring variables, as there is more about programming than just making it work correctly: well written programs work correctly as well as badly written programs (everything else is NOT a program but a mess) but a well written program will also be easy to understand, therefore easy to maintain and well documented (this plays up the same alley as "easy to understand"). What is "well written" versus "badly written" shows not always immediately but usually when trying to create version 10 from version 9. This is true for every programming language and every programming paradigma. Whatever the technology you use: strive to write well-written programs.

To declare variables has several values: first, you can assign sensible initial values so that you lessen the risk of using a variable before you are ready to use it. Second, declaring the variables up front builds automatically a sort-of "data dictionary" so that you quasi-automatically describe to yourself what you are using to variable for, what it is expect to hold, etc.. You build up a common point of reference that way and once your script became several hundred lines long and you have just used it for some months instead of rewrite it you will be thankful to yourself for the explanations to your thoughts you have left.

Programming is about organising your thoughts and the better you organise them the better the results are.

Yes, "-i" is for integer. Regarding the hiccup between "local" and "declare": probably "declare" is correct. I don't use bash much, so you can safely assume that to be my error, not yours.

I hope this helps.

bakunin