Shell script filtering


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script filtering
# 1  
Old 11-25-2014
Shell script filtering

Hi,

I have a file contains log information per line:
Code:
user=test1 group=test1 ctime=1411615875 qtime=1411615875 etime=1411615875 start=1411615876 resources_used.mem=390792kb resources_used.vmem=8844720kb resources_used.walltime=8

How can extract only user=test1 and resources_used.walltime=8:

Output:
Code:
user=test1 resources_used.walltime=8
user=test1 resources_used.walltime=9
user=test2 resources_used.walltime=6
...........

Thank you.

Kind regards,

- hce

Last edited by Franklin52; 11-27-2014 at 07:32 AM.. Reason: Please use code tags
# 2  
Old 11-25-2014
If user is always the first field and resources_used.walltime is always the last, then:
Code:
sed -e 's/ .* / /'

will do the job.

---------- Post updated at 06:18 AM ---------- Previous update was at 06:12 AM ----------

If the fields are in random order, this awk script will work:
Code:
{
  U = R = "";
  for (i = 1; i <= NF; i++) { 
     if ($i ~ /^user=/) U = $i;
     if ($i ~ /^resources_used.walltime=/) R = $i;
  }
  print U, R;
}

This User Gave Thanks to derekludwig For This Post:
# 3  
Old 11-25-2014
Code:
$ cat infile
user=test1 group=test1 ctime=1411615875 qtime=1411615875 etime=1411615875 start=1411615876 resources_used.mem=390792kb resources_used.vmem=8844720kb resources_used.walltime=8

Code:
awk -F'[ =]' '{s="";for(i=1;i<=NF;i+=2)if($i ~ /user|resources_used.walltime/){s = s ( s ? OFS :"") $i "=" $(i+1)}; if(s)print s }' infile
user=test1 resources_used.walltime=8

Code:
$ cat test.awk
BEGIN{
        # Add or remove field if required
	parse="user,resources_used.walltime"
	n=split(parse,P,/,/)
}
{
	s = ""
	for(i=1; i<=n; i++)
	{
		s  = s ( s ? OFS : "") ( match($0,P[i]"=[^ ]*") ? substr($0,RSTART,RLENGTH) : "NaN" )
	}
	if(s)print s
}

Code:
$ awk -f test.awk infile
user=test1 resources_used.walltime=8

This User Gave Thanks to Akshay Hegde For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to write config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

2. UNIX for Dummies Questions & Answers

How to write Config shell script to pass variables in master shell script?

Dear Unix gurus, We have a config shell script file which has 30 variables which needs to be passed to master unix shell script that invokes oracle database sessions. So those 30 variables need to go through the database sessions (They are inputs) via a shell script. one of the variable name... (1 Reply)
Discussion started by: dba1981
1 Replies

3. Shell Programming and Scripting

Help with shell script filtering IPs

Hello gentlemen. I would like to create a shell script (no perl or python please) to generate a list with those rules. Let's suppose I've this text file: a@A:soss(z)1.1.1.1 Opt!o:2.1.9.55 Azxk<ji>rC211.111.9.0-251.11.34.9 d=211.9.1.3 O.Oox 2.1.2.4-51.9.1.33... (6 Replies)
Discussion started by: accolito
6 Replies

4. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

5. Shell Programming and Scripting

Need help for filtering a file through awk script

Hello Folks, I am working on filtering a file having some special characters. Let's say for an example a file contains person name and phone number based on positions. First 5 characters name and next 10 characters phone number. My task is to , if there is any special character in phone number... (6 Replies)
Discussion started by: dinesh1985
6 Replies

6. Shell Programming and Scripting

script for automatic table filtering

Hello everyone! i I'm trying to write a script that can filter data in a text file. the source file looks like this the result file should look like this As you can see mostly i need to delete fields like _219402757693 and date - time 12.07.2012 8:49:06 i know that it's possible to... (0 Replies)
Discussion started by: o28
0 Replies

7. Programming

need help with shell script filtering files and sort! newbie question?

Hi folks, I would like to get familiar with shell script programing. The first task is: write a shell script that: scans your home-folder + sub-directory for all txt-files that all users of your group are allowed to read and write then output these files sorted by date of last... (4 Replies)
Discussion started by: rollinator
4 Replies

8. UNIX for Dummies Questions & Answers

Filtering using Shell Script

Hello, I am writing a small script which I want to get the values from several files and output the hits result, i.e. week1.hits includes the following, Mon Feb 4 22:02:35 GMT 2008 Tue Feb 5 23:22:12 GMT 2008 Sat Feb 9 01:56:16 GMT 2008 A sample of my script is this, #!/bin/bash ... (0 Replies)
Discussion started by: nitrous642
0 Replies

9. Shell Programming and Scripting

Filtering the yesterdays date from log files via script.

hi All, I have this sample text file - access.log: Jan 18 21:34:29 root 209.151.232.70 Jan 18 21:34:40 root 209.151.232.70 Jan 18 21:34:43 root 209.151.232.70 Jan 18 21:34:56 root 209.151.232.70 Jan 18 21:35:10 root 209.151.232.70 Jan 18 21:35:23 root 209.151.232.70 Jan 18 21:36:04 root... (2 Replies)
Discussion started by: linuxgeek
2 Replies

10. Shell Programming and Scripting

Issues with filtering duplicate records using gawk script

Hi All, I have huge trade file with milions of trades.I need to remove duplicate records (e.g I have following records) 30/10/2009,trdeId1,..,.. 26/10/2009.tradeId1,..,..,, 30/10/2009,tradeId2,.. In the above case i need to filter duplicate recods and I should get following output.... (2 Replies)
Discussion started by: nmumbarkar
2 Replies
Login or Register to Ask a Question