Wget in bash using sed and awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget in bash using sed and awk
# 1  
Old 05-01-2015
Wget in bash using sed and awk

In the bash below when the program is opened the download function runs and downloads the getCSV file and on the screen "Downloading getCSV.csv:%" displays and when it completes the menu function is called. However, as of now the bash opens and closes after a few seconds and I'm not sure why. Thank you Smilie.


Code:
#!/bin/bash

download () {
    url=http://xxx.xx.xxx.xxx/data/getCSV.csv
    echo -n "    "
    wget -O getCSV.txt --progress=dot $url 2>&1 | grep --line-buffered "%" | \
        sed -u -e "s,\.,,g" | awk '{printf("\b\b\b\b%4s", $2)}'
    echo -ne "\b\b\b\b"
    echo " DONE"
menu
}

menu() {
    while true
    do
        printf "\n Welcome to NGS menu (v1), please make a selection from the MENU \n
        ==================================\n\n
        \t 1  Patient QC\n
        ==================================\n\n"

        printf "\t Your choice: "; read menu_choice

        case "$menu_choice" in
        1) patient ;;
        *) printf "\n Invalid choice."; sleep 2 ;;
        esac
    done
}

# 2  
Old 05-01-2015
make your last line call download. right now you just define two functions and don't start any.
# 3  
Old 05-01-2015
I'm not sure I follow in my other bash menus I follow the same structure posted, but it doesn't seem to be working for this. How would you code this (I am probabaly not doing it as I should). Thank you Smilie.
# 4  
Old 05-01-2015
Why not wget --quiet --show-progress instead of the grep, sed, and awk?
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 05-01-2015
Thanks I will work on a new bash using that Smilie.

---------- Post updated at 01:30 PM ---------- Previous update was at 12:53 PM ----------

In the below bash is it possible to display the progress of the download then when it completes echo "download complete....

Also, when the user inputs an id to search for the bash closes and if a match is found outputs a new file (match.txt). Is it possible to have not close the bash but rather, on the screen "searching for match" and if a match is found "match found in line.." is displayed and written to match.txt else "no match found" is displayed. Thank you Smilie.

Code:
#!/bin/bash

cd 'C:\Users\cmccabe\Desktop\wget'
       wget -O getCSV.txt http://xxx.xx.xxx.xxx/data/getCSV.csv --quiet

printf "downloading file, please wait"
echo "download complete, what is the id of the NGS patient: "; read id"

	   [ -z "$id" ] && printf "\n No ID supplied. Leaving match function." && sleep 2 && return
    [ "$id" = "end" ] && printf "\n Leaving match function." && sleep 2 && return
	   
input=$id    
while read -r line
do
if [ -n "$id" ]; then
echo "match found in line $id"|sed 's/:.*//'
echo "$id"|sed 's/[^:]*://' >> match.txt
else
echo "no match found"
done

I think curl will address my first problem, but I'm still not sure about the bash . Why does it close after the user enters an id?

Last edited by cmccabe; 05-01-2015 at 06:00 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Store filenames for wget in bash

I have a bash that downloads a list of files as a text file using wget. What I now need to do is store those files names and pass them to a download call also using wget. List.txt in /home directory FilterDuplicates.html file1.bam file2.bam file3.bam file1.vcf.gz file2.vcf.gz... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Text manipulation with sed/awk in a bash script

Guys, I have a variable in a script that I want to transform to into something else Im hoping you guys can help. It doesn't have to use sed/awk but I figured these would be the simplest. DATE=20160120 I'd like to transform $DATE into "01-20-16" and move it into a new variable called... (8 Replies)
Discussion started by: dendenyc
8 Replies

3. Shell Programming and Scripting

Wget in bash

I am attempting to write a bash that starts by using wget and getting the following errors: Stand-alone code that works: wget -O getCSV.txt http://172.24.xxx.xxx/data/getCSV.csv c:\cygwin\home\cmccabe\NGS.sh: line 2: $'\r': command not found : No such file or directorysh: line 3:... (2 Replies)
Discussion started by: cmccabe
2 Replies

4. Shell Programming and Scripting

Search strings and highlight them using Perl or bash/awk/sed

Hi, I have two files: a.doc and b.txt I wish to search the strings from file b.txt in a.doc and want to highlight them in a.doc with different colours using Perl or bash./awk/sed? Please guide me. :) Thanks!!!!! (10 Replies)
Discussion started by: bioinfo
10 Replies

5. Shell Programming and Scripting

Rsync script to rewrite suffix - BASH, awk, sed, perl?

trying to write up a script to put the suffix back. heres what I have but can't get it to do anything :( would like it to be name.date.suffix rsync -zrlpoDtub --suffix=".`date +%Y%m%d%k%M%S`.~" --bwlimit=1024 /mymounts/test1/ /mymounts/test2/ while IFS=. read -r -u 9 -d '' name... (1 Reply)
Discussion started by: jmituzas
1 Replies

6. Shell Programming and Scripting

XML- Sed || Awk Bash script... Help!

Hi ! I'm working into my first bash script to make some xml modification and it's going to make me crazy lol .. so I decide to try into this forum to take some ideas from people that really know about this! This is my situation I've and xml file with a lots of positional values with another tags... (9 Replies)
Discussion started by: juampal
9 Replies

7. Shell Programming and Scripting

bash variable (set via awk+sed) not working as expected

Hi! Been working on a script and I've been having a problem. I've finally narrowed it down to this variable I'm setting: servername=$(awk -v FS=\/ '{ print $7 } blah.txt | sed 's\/./-/g' | awk -v FS=\- '{print $1}')" This will essentially pare down a line like this: ... (7 Replies)
Discussion started by: creativedynamo
7 Replies

8. Shell Programming and Scripting

Replace last row of a column in bash/awk/sed

Hi, I've got a file with 3 columns which ends like this: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 1400 I want to replace the last row of the last column with the value 0. So my new file will end: ... 1234 345 1400 5287 733 1400 8472 874 1400 9317 726 ... (5 Replies)
Discussion started by: jhunter87
5 Replies

9. Shell Programming and Scripting

using sed on bash variables (or maybe awk?)

Hi all- I've been fooling with this for a few days, but I'm rather new at this... I have a bash variable containing a long string of various characters, for instance: JUNK=this that the other xyz 1234 56 789 I don't know what "xyz" actually is, but I know that: START=he other and ... (2 Replies)
Discussion started by: rev66
2 Replies

10. Shell Programming and Scripting

How to script wget in bash?

The script below is giving me grief! The error message says /download.bash: line 16: syntax error near unexpected token `else' ./download.bash: line 16: `else wget "http://downloads.sourceforge.net/hibernate/hibernate-3.2.5.ga.zip?modtime=1185893922&big_mirror=1" ' I think it must be a... (1 Reply)
Discussion started by: siegfried
1 Replies
Login or Register to Ask a Question