Sponsored Content
Full Discussion: Noob trying to improve
Operating Systems OS X (Apple) Noob trying to improve Post 302988248 by Ardzii on Thursday 22nd of December 2016 11:17:34 AM
Old 12-22-2016
Code Noob trying to improve

Hi everyone!

This is my first post here, I hope that I will not already be violating any rule! I also would like to apologize in advance as my post will definitely be a noob post... please have patience and faith Smilie!

Now that I have set the ground rules SmilieSmilie, my objective is trying to understand how to write a sort of script which could search and extract information out of the web and put that information into a CSV file.

To be completely transparent, I sort of already have a specific idea of what kind of information I'd like to get: The prices and details of some asset listings on the web.

I have checked the web and found out that with a while loop with a bunch of curl and grep commands could do the trick. Is there anyone who can help me in building this sort of "web-crawler".

Thanks in advance to you all!

Ardzii
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can I improve this script ???

Hi all, Still a newbie and learning as I go ... as you do :) Have created this script to report on disc usage and I've just included the ChkSpace function this morning. It's the first time I've read a file (line-by-bloody-line) and would like to know if I can improve this script ? FYI - I... (11 Replies)
Discussion started by: Cameron
11 Replies

2. UNIX for Advanced & Expert Users

improve performance by using ls better than find

Hi , i'm searching for files over many Aix servers with rsh command using this request : find /dir1 -name '*.' -exec ls {} \; and then count them with "wc" but i would improve this search because it's too long and replace directly find with ls command but "ls *. " doesn't work. and... (3 Replies)
Discussion started by: Nicol
3 Replies

3. Shell Programming and Scripting

improve this?

Wrote this script to find the date x days before or after today. Is there any way that this script can be speeded up or otherwise improved? #!/usr/bin/sh check_done() { if then daysofmth=31 elif then if ... (11 Replies)
Discussion started by: blowtorch
11 Replies

4. UNIX for Dummies Questions & Answers

Improve Performance

hi someone tell me which ways i can improve disk I/O and system process performance.kindly refer some commands so i can do it on my test machine.thanks, Mazhar (2 Replies)
Discussion started by: mazhar99
2 Replies

5. Shell Programming and Scripting

Any way to improve performance of this script

I have a data file of 2 gig I need to do all these, but its taking hours, any where i can improve performance, thanks a lot #!/usr/bin/ksh echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')" function showHelp { cat << EOF >&2 syntax extreme.sh FILENAME Specify filename to parse EOF... (3 Replies)
Discussion started by: sirababu
3 Replies

6. IP Networking

How to improve throughput?

I have a 10Gbps network link connecting two machines A and B. I want to transfer 20GB data from A to B using TCP. With default setting, I can use 50% bandwidth. How to improve the throughput? Is there any way to make throughput as close to 10Gbps as possible? thanks~ :) (3 Replies)
Discussion started by: andrewust
3 Replies

7. Shell Programming and Scripting

Want to improve the performance of script

Hi All, I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately. Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies

8. AIX

improve sulog

I just wrote a very small script that improves readability on system sulog. The problem with all sulog is there is lack of clarity whether the info you are looking at is the most current. So if you just need a simple soution instead of going thru the trouble of writing a script that rotate logs and... (0 Replies)
Discussion started by: sparcguy
0 Replies

9. Shell Programming and Scripting

How to improve an script?

Gents. I have 2 different scripts for the same purpose: raw2csv_1 Script raw2csv_1 finish the process in less that 1 minute raw2csv_2 Script raw2csv_2 finish the process in more that 6 minutes. Can you please check if there is any option to improve the raw2csv_2. To finish the job... (4 Replies)
Discussion started by: jiam912
4 Replies

10. Shell Programming and Scripting

Improve script

Gents, Is there the possibility to improve this script to be able to have same output information. I did this script, but I believe there is a very short code to get same output here my script awk -F, '{if($10>0 && $10<=15) print $6}' tmp1 | sort -k1n | awk '{a++} END { for (n in a )... (23 Replies)
Discussion started by: jiam912
23 Replies
GIT-WEB--BROWSE(1)						    Git Manual							GIT-WEB--BROWSE(1)

NAME
git-web--browse - git helper script to launch a web browser SYNOPSIS
git web--browse [OPTIONS] URL/FILE ... DESCRIPTION
This script tries, as much as possible, to display the URLs and FILEs that are passed as arguments, as HTML pages in new tabs on an already opened web browser. The following browsers (or commands) are currently supported: o firefox (this is the default under X Window when not using KDE) o iceweasel o seamonkey o iceape o chromium (also supported as chromium-browser) o google-chrome (also supported as chrome) o konqueror (this is the default under KDE, see Note about konqueror below) o opera o w3m (this is the default outside graphical environments) o elinks o links o lynx o dillo o open (this is the default under Mac OS X GUI) o start (this is the default under MinGW) Custom commands may also be specified. OPTIONS
-b <browser>, --browser=<browser> Use the specified browser. It must be in the list of supported browsers. -t <browser>, --tool=<browser> Same as above. -c <conf.var>, --config=<conf.var> CONF.VAR is looked up in the git config files. If it's set, then its value specifies the browser that should be used. CONFIGURATION VARIABLES
CONF.VAR (from -c option) and web.browser The web browser can be specified using a configuration variable passed with the -c (or --config) command line option, or the web.browser configuration variable if the former is not used. browser.<tool>.path You can explicitly provide a full path to your preferred browser by setting the configuration variable browser.<tool>.path. For example, you can configure the absolute path to firefox by setting browser.firefox.path. Otherwise, git web--browse assumes the tool is available in PATH. browser.<tool>.cmd When the browser, specified by options or configuration variables, is not among the supported ones, then the corresponding browser.<tool>.cmd configuration variable will be looked up. If this variable exists then git web--browse will treat the specified tool as a custom command and will use a shell eval to run the command with the URLs passed as arguments. NOTE ABOUT KONQUEROR
When konqueror is specified by a command line option or a configuration variable, we launch kfmclient to try to open the HTML man page on an already opened konqueror in a new tab if possible. For consistency, we also try such a trick if browser.konqueror.path is set to something like A_PATH_TO/konqueror. That means we will try to launch A_PATH_TO/kfmclient instead. If you really want to use konqueror, then you can use something like the following: [web] browser = konq [browser "konq"] cmd = A_PATH_TO/konqueror Note about git-config --global Note that these configuration variables should probably be set using the --global flag, for example like this: $ git config --global web.browser firefox as they are probably more user specific than repository specific. See git-config(1) for more information about this. GIT
Part of the git(1) suite Git 1.7.10.4 11/24/2012 GIT-WEB--BROWSE(1)
All times are GMT -4. The time now is 02:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy