Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sending output of program into subsequent commands (i.e. awk/sort)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sending output of program into subsequent commands (i.e. awk/sort)
# 1  
Old 06-25-2019
Sending output of program into subsequent commands (i.e. awk/sort)
Hi,

I have identified how to use command chaining as per below on a file, to capture the header of a file, as well as the line containing the C: drive.

Code:
$ cat test.txt
Filesystem      Size  Used Avail Use% Mounted on
rootfs          237G  153G   84G  65% /
none            237G  153G   84G  65% /dev
none            237G  153G   84G  65% /run
none            237G  153G   84G  65% /run/lock
none            237G  153G   84G  65% /run/shm
none            237G  153G   84G  65% /run/user
C:              237G  153G   84G  65% /mnt/c

$ cat <(head -n1 test.txt) <(awk '/C/' test.txt)
Filesystem      Size  Used Avail Use% Mounted on
C:              237G  153G   84G  65% /mnt/c

I am trying to use the same logic on the output of a command, but it does not work. How can I get this to work?

Aim is to sort docker images by size column on the right, while printing the first line (header).

Code:
$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
alpine              latest              4d90542f0623        5 days ago          5.58MB
ubuntu              xenial              13c9f1285025        6 days ago          119MB
ubuntu              latest              4c108a37151f        6 days ago          64.2MB
centos              latest              9f38484d220f        3 months ago        202MB
linode/lamp         latest              2359fa12fded        4 years ago         372MB

$ docker image ls | awk 'NR>1' | sort -k5 -V -r
linode/lamp         latest              2359fa12fded        4 years ago         372MB
centos              latest              9f38484d220f        3 months ago        202MB
ubuntu              xenial              13c9f1285025        6 days ago          119MB
ubuntu              latest              4c108a37151f        6 days ago          64.2MB
alpine              latest              4d90542f0623        5 days ago          5.58MB

$ docker image ls | (awk 'NR==1') <(sort -k5 -V -r)
-bash: syntax error near unexpected token `<(sort -k5 -V -r)'  <----- failure
$ docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
alpine              latest              4d90542f0623        5 days ago          5.58MB
ubuntu              xenial              13c9f1285025        6 days ago          119MB
ubuntu              latest              4c108a37151f        6 days ago          64.2MB
centos              latest              9f38484d220f        3 months ago        202MB
linode/lamp         latest              2359fa12fded        4 years ago         372MB

Thanks
Last edited by sand1234; 06-29-2019 at 11:52 PM..
# 2  
Old 06-25-2019
Not sure what you're after. Header plus a sorted list of the residual lines? How about
Code:
docker image ls | awk 'NR==1 {print; next} {print | "sort -Vrk5"}'

Be aware that sort sees the "days/months/years" field as its key 5.
Last edited by RudiC; 06-25-2019 at 04:09 AM..
This User Gave Thanks to RudiC For This Post:
sand1234
# 3  
Old 06-25-2019
RudiC,
This works perfectly:

Code:
$ docker image ls | awk 'NR==1 {print; next} {print | "sort -Vrk5"}'
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
linode/lamp         latest              2359fa12fded        4 years ago         372MB
centos              latest              9f38484d220f        3 months ago        202MB
ubuntu              xenial              13c9f1285025        6 days ago          119MB
ubuntu              latest              4c108a37151f        6 days ago          64.2MB
alpine              latest              4d90542f0623        5 days ago          5.58MB

Is there any way to use command chaining like I used previously for this example? i.e. multiple redirections.
Quote:
cat <(head -n1 test.txt) <(awk '/C/' test.txt)
# 4  
Old 06-25-2019
Not with a single input source piped into something. Note that you open and read test.txt twice in your above construct. You could do (but this is far from being smart nor efficient)


Code:
cat <(docker image ls | head -n1) <(docker image ls | tail -n+2 | sort -Vrk5 )

You might feel inspired to use tee to split the input into different branches of execution, but it'd be difficult to predict the resultant order of lines. This clumsy approach seems to do it on my system
Code:
docker image ls | tee >(head -n1) >(tail -n+2 | sort -rk5) > /dev/null

, but, again, this is far from using common sense.
This User Gave Thanks to RudiC For This Post:
sand1234
# 5  
Old 06-29-2019
shell commands and awk
Hi RudiC,

I find it interesting that we are able to push shell commands directly into awk.
Is there any documentation around how this works? Is this only supported with the 'print' statement?

e.g.
Code:
awk 'NR==1 {print; next} {print | "sort -Vrk5"}'

Thanks.
# 6  
Old 06-30-2019
Yes, man awk, amongst others:
Quote:
The output of print and printf can be redirected to a file or command by appending > file, >> file or | command to the end of the print statement. Redirection opens file or command only once, subsequent redirections append to the already open stream.
It is a standard feature of awk for both print and printf.
This User Gave Thanks to RudiC For This Post:
sand1234
# 7  
Old 06-30-2019
Hi RudiC,

Thanks, interesting to know.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk Sort 2d histogram output from min(X,Y) to max(X,Y)

I've got Gnuplot-format 2D histogram data output which looks as follows. 6.5 -1.25 10.2804 6.5404 -1.25 10.4907 6.58081 -1.25 10.8087 6.62121 -1.25 10.4686 6.66162 -1.25 10.506 6.70202 -1.25 10.3084 6.74242 -1.25 9.68256 6.78283 -1.25 9.41229 6.82323 -1.25 9.43078 6.86364 -1.25 9.62408... (1 Reply)
Discussion started by: chrisjorg
1 Replies

2. Shell Programming and Scripting

awk capturing first sample, but not subsequent id's

In the awk below the first sample MEV45 gets extracted from the html, but the subsequent MEV46 and MEV47 do not as they are not part of parse. I can not seem to add them to the code. Thank you very much @RudiC your awk is very nice :). input {"barcodeId": "IonXpress", "barcodedSamples":... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

Pipe or combine output of three awk commands

What is the correct syntax to pipe or run three awk commands? Basically, using the output of the first awk as input in the second. Then using the output of the second awk in the third. Thank you :). awk 'FNR==NR {E; next }$3 in E {print $3, $5}' panel_genes.txt RefSeqGene.txt > update.txt |... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Want to sort a file using awk & sed to get required output

Hi All, Need Suggestion, Want to sort a file using awk & sed to get required, output as below, such that each LUN shows correct WWPN and FA port Numbers correctly: Required output: 01FB 10000000c97843a2 8C 0 01FB 10000000c96fb279 9C 0 22AF 10000000c97843a2 8C 0 22AF 10000000c975adbd ... (10 Replies)
Discussion started by: aix_admin_007
10 Replies

5. UNIX for Dummies Questions & Answers

is there a way to assing variable a value that is output of a program in awk script?

Hi there is there a way to assing variable a value that is output of a program in awk script. For e.g., I did temp=(`grep "" $5 | cut -f8 -d' '`) but it does not work. Any advice??? Thanks in advance!!! :) (3 Replies)
Discussion started by: FUTURE_EINSTEIN
3 Replies

6. Shell Programming and Scripting

awk - use fields from subsequent lines

I've run into a problem getting exactly what I want out of awk - some folks may recognize this as an output from Amazon's ec2-describe-instances: Given the following: INSTANCE i-4960f321 BLOCKDEVICE Line2Var2 TAG instance i-4960f321 Name web1 TAG instance i-4960f321... (2 Replies)
Discussion started by: colinjohnson
2 Replies

7. Shell Programming and Scripting

awk text record - prepend first field to all subsequent fields

Hello everyone, I've suddenly gotten very interested in sed and awk (and enjoying it quite a bit too) because of a large conversion project that we're working on. I'm currently stuck with a very inefficient process for processing text blocks. I'm sure someone here should be able to easily point out... (2 Replies)
Discussion started by: jameswatson3
2 Replies

8. UNIX for Dummies Questions & Answers

Sending awk output to nothing?

How can I direct awk output to go nowhere? I can write the data to a file easy enough or print it on the screen. But for this particular command I don't actually want the data and I don't want to create a file. I just want the data discarded. Thanks in advance (3 Replies)
Discussion started by: MrEddy
3 Replies

9. UNIX for Dummies Questions & Answers

Using the Sort, Cut, and Awk commands

Hello, I am trying, utilizing the few commands I know, to extract all records within my file that were generated in November of 2007. Each record within the file has a "date" field giving the month, day, and year (9-8-88). How do I extract those records to their own file? Once I extract... (4 Replies)
Discussion started by: babbabooey
4 Replies

10. Shell Programming and Scripting

combining unix commands and awk program

Dear Experts I am trying to find if it is possible to combine unix commands in awk program. For example if it is possible embed rm or ls or any unix command inside the awk program and while it is reading the file besides printing be able to do some unix commands. I am thinking may be just print... (2 Replies)
Discussion started by: Reza Nazarian
2 Replies
Login or Register to Ask a Question