Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Wget, grep, sort, sed in 1 command/script Post 302996134 by p1ne on Wednesday 19th of April 2017 02:39:16 PM
Old 04-19-2017
Wget, grep, sort, sed in 1 command/script

Hi, I need to join these statements for efficiency, and without having to make a new directory for each batch. I'm annotating commands below.
Code:
wget -q -r -l1 URL 
^^ can't use -O - here and pipe | to grep because of -r
grep -hrio "\b[a-z0-9.-]\+@[a-z0-9.-]\+\.[a-z]\{2,4\}\+\b" * > first.txt
^^ Need to grep the output of wget only; at present it's grepping other files in the directory.
sort -u first.txt > second.txt && sed '0~5 a\\' second.txt > third.txt
^^ piping | these doesn't work; && does.

Thanks in advance for direction.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort script/command

ok. i am doing a project where i have hand typed in the titles of nearly 500 DVD titles, each one is on a seperate line. but they arent in any type of alphebetical order, and i need them sorted in that format (A-Z or a-z) ..... i know that the 'sort' command can be used but also know the... (6 Replies)
Discussion started by: Chadbot
6 Replies

2. Shell Programming and Scripting

Complex find grep or sed command

Haven't worked in bash for ages. did a good bit of shell scripting in regular sh, but have forgotten most of it. I have several thousand php files that now include the following line at the end of the file. There is no LF or CR/LF before it begins, it is just concatenated to the final line of... (3 Replies)
Discussion started by: sjburden
3 Replies

3. UNIX for Dummies Questions & Answers

Using grep output as input for sed command

Hi, I would like to know if this is possible, and if so what can i do to make this work. I would like to grep a line X from fileA and then use the output to replace a word Y in fileB. grep "line X" fileA | sed -e 's/Y/X/g' > outfile this statement does not work, as i do not know how to... (7 Replies)
Discussion started by: cavanac2
7 Replies

4. Shell Programming and Scripting

Help with grep awk sed command

I have a txt file with data abc:def:ghi:jkl:mno pq stu vwx I want to take out abc what should i do? I try awk awk '/abc/ {print $1}' listfile.txt> extendedfile.txt (5 Replies)
Discussion started by: Learnerabc
5 Replies

5. Shell Programming and Scripting

Shell script with wget in ssh command

Hi, I am using a linux with bash. I have a script written which will login to a remote server and from there it runs a "wget" to downlaod a build file from a webserver. Here is the line inside the script: ssh -t -q -o StrictHostKeyChecking=no -o ConnectTimeout=5 root@${a}'wget... (4 Replies)
Discussion started by: sunrexstar
4 Replies

6. Shell Programming and Scripting

Using a combination of sort/cut/grep/awk/join/paste/sed

I have a file and need to only select users that have a shell of “/bin/bash” in the line using awk or sed please help (4 Replies)
Discussion started by: boyboy1212
4 Replies

7. Shell Programming and Scripting

Deleting lines from wget script using find/sed

I am downloading numerous files (600 plus) using a 'wget' script. Some files are not downloading and have zero byte size. I am using the following 'find' command to find the files in my cd which have non-zero byte size after the wget script has been run. find -type f -size +0 -exec basename {}... (3 Replies)
Discussion started by: dl226
3 Replies

8. Shell Programming and Scripting

applescript & grep - sed command

I'm new using Unix commands in applescript. The following script you choose different folders with PDfs, get file count of PDfs on chosen folders, & write the results in text file. set target_folder to choose folder with prompt "Choose target folders containing only PDFs to count files" with... (0 Replies)
Discussion started by: nellbern
0 Replies

9. Programming

[ask]SQL command act like sort and grep

for example, I have a text file in random content inside, maybe something like this. 234234 54654 123134 467456 24234234 7867867 23424 568567if I run this command cat "filename.txt" | sort -n | grep "^467456$" -A 1 -B 1the result is 234234 467456 568567is it possible to do this command... (2 Replies)
Discussion started by: 14th
2 Replies
SHTOOL-SUBST.TMP(1)					      GNU Portable Shell Tool					       SHTOOL-SUBST.TMP(1)

NAME
shtool-subst - GNU shtool sed(1) substitution operations SYNOPSIS
shtool subst [-v|--verbose] [-t|--trace] [-n|--nop] [-w|--warning] [-q|--quiet] [-s|--stealth] [-i|--interactive] [-b|--backup ext] [-e|--exec cmd] [-f|--file cmd-file] [file] [file ...] DESCRIPTION
This command applies one or more sed(1) substitution operations to stdin or any number of files. OPTIONS
The following command line options are available. -v, --verbose Display some processing information. -t, --trace Enable the output of the essential shell commands which are executed. -n, --nop No operation mode. Actual execution of the essential shell commands which would be executed is suppressed. -w, --warning Show warning on substitution operation resulting in no content change on every file. The default is to show a warning on substitution operations resulted in no content change on all files. -q, --quiet Suppress warning on substitution operation resulting in no content change. -s, --stealth Stealth operation. Preserve timestamp on file. -i, --interactive Enter interactive mode where the user has to approve each operation. -b, --backup ext Preserve backup of original file using file name extension ext. Default is to overwrite the original file. -e, --exec cmd Specify sed(1) command directly. -f, --file cmd-file Read sed(1) command from file. EXAMPLE
# shell script shtool subst -i -e 's;(c) ([0-9]*)-2000;(c) 1-2001;' *.[ch] # RPM spec-file %install shtool subst -v -n -e 's;^(prefix=).*;1 $RPM_BUILD_ROOT%{_prefix};g' -e 's;^(sysconfdir=).*;1 $RPM_BUILD_ROOT%{_prefix}/etc;g' `find . -name Makefile -print` make install HISTORY
The GNU shtool subst command was originally written by Ralf S. Engelschall <rse@engelschall.com> in 2001 for GNU shtool. It was prompted by the need to have a uniform and convenient patching frontend to sed(1) operations in the OpenPKG package specifications. SEE ALSO
shtool(1), sed(1). 18-Jul-2008 shtool 2.0.8 SHTOOL-SUBST.TMP(1)
All times are GMT -4. The time now is 02:20 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy