Shell script / Grep / Awk to variable and Loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script / Grep / Awk to variable and Loop
# 1  
Old 01-26-2010
Question Shell script / Grep / Awk to variable and Loop

Hi,

I have a text file with data in that I wish to extract, assign to a variable and process through a loop.

Kind of the process that I am after:

1: Grep the text file for the values.
Currently using:
cat /root/test.txt | grep TESTING= | awk -F"=" '{ a[i++] = $2 } {print a[1]}' | sort -u

This will print out the first value. (Change the print to then print the second, print a[2] )

2: Assign all occurences of my extracted value to a variable.

3: With each occurrence/ variable pass it through a loop and use it to locate/extract more information.

4: The information will then be output to another file.

For example.

Test.txt content:

Hello Mum
TESTING=FortyFive
Hello World
TESTING=IO432

Assign both grep'd/awk'd data to a variable, because for each it needs to loop through more grep/awk's to find more data.

I want to extract and display in my output file:

TESTING::FortyFive,IO432
TESTED::More extracted data,Even more data
AGAIN::Still more, and again


Therefore the data before the comma related to the first value, FortyFive, the data after the comma in the output file refers to the second value, IO432.

I hope you can understand what I am trying to do.

Hope someone can help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script use two variable in for loop

I want to create a shell script to add a user and modify its comment field mentioned in a file. 1. File value:- username comment field value xyz123 xyztesting abc123 abctesting def123 deftesting 2. i am using below loop to create user... (2 Replies)
Discussion started by: Anil
2 Replies

2. UNIX for Beginners Questions & Answers

How can I assign awk's variable to shell script's variable?

I have the following script, and I want to assign the output ($10 and $5) from awk to N and L: grdinfo data.grd | awk '{print $10,$5}'| read N L output from gridinfo data.grd is: data.grd 50 100 41 82 -2796 6944 0.016 0.016 3001 2461. where N and L is suppose to be 3001 and 100. I use... (8 Replies)
Discussion started by: geomarine
8 Replies

3. UNIX for Beginners Questions & Answers

How to grep variable in shell script?

Hi Team, I am trying to grep date in a file but it is not working #!/bin/bash d=`date "+ %Y%m%d %H:%M"` cd /scripts /bin/rm -f test1 cat /var/logs/File.txt.0 |grep $"d" >v.txt instead it is showing the complete file output. kindly suggest how should i grep this variable it works... (7 Replies)
Discussion started by: scriptor
7 Replies

4. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

5. Shell Programming and Scripting

Does a variable lose its value outside the loop in shell script?

hi, when we assign a variable inside a for loop or while loop in a shell script, does it loses its value after comming out of the loop. i am facing this issue. can anyone help me?? (8 Replies)
Discussion started by: Little
8 Replies

6. Shell Programming and Scripting

Issue with passing variable to Grep in a shell script

Hi, I'm trying to check if methods specified in a class have been added to the corrosponding interface. My code below is giving me the following errors: grep: function: No such file or directory grep: import($zipfile): No such file or directory grep: function: No such file or... (1 Reply)
Discussion started by: racshot65
1 Replies

7. Shell Programming and Scripting

How to give a variable output name in a shell script inside a for loop

Hi all I run my program prog.c in the following way : $ ./prog 1 > output.txt where 1 is a user defined initial value used by the program. But now I want to run it for many a thousand initial values, 1-1000, and store all the outputs in different files. Like $ ./prog 1... (1 Reply)
Discussion started by: alice06
1 Replies

8. Shell Programming and Scripting

assign awk's variable to shell script's variable?

Dear All, we have a command output which looks like : Total 200 queues in 30000 Kbytes and we're going to get "200" and "30000" for further process. currently, i'm using : numA=echo $OUTPUT | awk '{print $2}' numB=echo $OUTPUT | awk '{print $5}' my question is : can I use just one... (4 Replies)
Discussion started by: tiger2000
4 Replies

9. Shell Programming and Scripting

Passing a variable to awk while in a shell for loop

I am a newbie to awk and c programming, however am not a unix newbie. However, I do need help with a kshell script I am writing. It is almost complete, the last step is killing me. Any help would be greatly appreciated. What I am trying to do is cat a text file that has usernames. Then, using... (2 Replies)
Discussion started by: synergy_texas
2 Replies

10. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies
Login or Register to Ask a Question
POE::Filter::Stackable(3pm)				User Contributed Perl Documentation			       POE::Filter::Stackable(3pm)

NAME
POE::Filter::Stackable - combine multiple POE::Filter objects SYNOPSIS
#!perl use POE qw( Wheel::FollowTail Filter::Line Filter::Grep Filter::Stackable ); POE::Session->create( inline_states => { _start => sub { my $parse_input_as_lines = POE::Filter::Line->new(); my $select_sudo_log_lines = POE::Filter::Grep->new( Put => sub { 1 }, Get => sub { my $input = shift; return $input =~ /sudo[d+]/i; }, ); my $filter_stack = POE::Filter::Stackable->new( Filters => [ $parse_input_as_lines, # first on get, last on put $select_sudo_log_lines, # first on put, last on get ] ); $_[HEAP]{tailor} = POE::Wheel::FollowTail->new( Filename => "/var/log/system.log", InputEvent => "got_log_line", Filter => $filter_stack, ); }, got_log_line => sub { print "Log: $_[ARG0] "; } } ); POE::Kernel->run(); exit; DESCRIPTION
POE::Filter::Stackable combines multiple filters together in such a way that they appear to be a single filter. All the usual POE::Filter methods work, but data is secretly passed through the stacked filters before it is returned. POE::Wheel objects and stand-alone programs need no modifications to work with a filter stack. In the "SYNOPSIS", POE::Filter::Line and POE::Filter::Grep are combined into one filter that only returns a particular kind of line. This can be more efficient than filtering lines in application space, as fewer events may need to be dispatched and handled. Internally, filters are stored in an array. Data added by get_one_start() will flow through the filter array in increasing index order. Filter #0 will have first crack at it, followed by filter #1 and so. The get_one() call will return an item after it has passed through the last filter. put() passes data through the filters in descending index order. Data will go through the filter with the highest index first, and put() will return the results after data has passed through filter #0. PUBLIC FILTER METHODS
In addition to the usual POE::Filter methods, POE::Filter::Stackable also supports the following. new By default, new() creates an empty filter stack that behaves like POE::Filter::Stream. It may be given optional parameters to initialize the stack with an array of filters. my $sudo_lines = POE::Filter::Stackable->new( Filters => [ POE::Filter::Line->new(), POE::Filter::Grep->new( Put => sub { 1 }, # put all items Get => sub { shift() =~ /sudo[d+]/i }, ), ] ); pop Behaves like Perl's built-in pop() for the filter stack. The highest-indexed filter is removed from the stack and returned. Any data remaining in the filter's input buffer is lost, but an application may always call "get_pending" in POE::Filter on the returned filter. my $last_filter = $stackable->pop(); my $last_buffer = $last_filter->get_pending(); shift Behaves like Perl's built-in shift() for the filter stack. The 0th filter is removed from the stack and returned. Any data remaining in the filter's input buffer is passed to the new head of the stack, or it is lost if the stack becomes empty. An application may also call "get_pending" in POE::Filter on the returned filter to examine the filter's input buffer. my $first_filter = $stackable->shift(); my $first_buffer = $first_filter->get_pending(); push FILTER[, FILTER] push() adds one or more new FILTERs to the end of the stack. The newly pushed FILTERs will process input last, and they will handle output first. # Reverse data read through the stack. # rot13 encode data sent through the stack. $stackable->push( POE::Filter::Map->( Get => sub { return scalar reverse shift() }, Put => sub { local $_ = shift(); tr[a-zA-Z][n-za-mN-ZA-M]; $_ }, ) ); unshift FILTER[, FILTER] unshift() adds one or more new FILTERs to the beginning of the stack. The newly unshifted FILTERs will process input first, and they will handle output last. filters filters() returns a list of the filters inside the Stackable filter, in the stack's native order. Calling "<$filter_stack-"filters()>> in the "SYNOPSIS" would return a list of two filter objects: POE::Filter::Line=ARRAY(0x8b5ee0) POE::Filter::Grep=ARRAY(0x8b5f7c) filter_types filter_types() returns a list of class names for each filter in the stack, in the stack's native order. Calling "<$filter_stack-"filter_types()>> in the "SYNOPSIS" would return a list of two class names: POE::FIlter::Line POE::Filter::Grep It could easily be replaced by: my @filter_types = map { ref } $filter_stack->filters; SEE ALSO
POE::Filter for more information about filters in general. Specific filters, amongst which are: POE::Filter::Block, POE::Filter::Grep, POE::Filter::HTTPD, POE::Filter::Line, POE::Filter::Map, POE::Filter::RecordBlock, POE::Filter::Reference, POE::Filter::Stream BUGS
None currently known. AUTHORS &; COPYRIGHTS The Stackable filter was contributed by Dieter Pearcey. Documentation provided by Rocco Caputo. Please see the POE manpage for more information about authors and contributors. perl v5.14.2 2012-05-15 POE::Filter::Stackable(3pm)