Shell script / Grep / Awk to variable and Loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script / Grep / Awk to variable and Loop
# 1  
Old 01-26-2010
Question Shell script / Grep / Awk to variable and Loop

Hi,

I have a text file with data in that I wish to extract, assign to a variable and process through a loop.

Kind of the process that I am after:

1: Grep the text file for the values.
Currently using:
cat /root/test.txt | grep TESTING= | awk -F"=" '{ a[i++] = $2 } {print a[1]}' | sort -u

This will print out the first value. (Change the print to then print the second, print a[2] )

2: Assign all occurences of my extracted value to a variable.

3: With each occurrence/ variable pass it through a loop and use it to locate/extract more information.

4: The information will then be output to another file.

For example.

Test.txt content:

Hello Mum
TESTING=FortyFive
Hello World
TESTING=IO432

Assign both grep'd/awk'd data to a variable, because for each it needs to loop through more grep/awk's to find more data.

I want to extract and display in my output file:

TESTING::FortyFive,IO432
TESTED::More extracted data,Even more data
AGAIN::Still more, and again


Therefore the data before the comma related to the first value, FortyFive, the data after the comma in the output file refers to the second value, IO432.

I hope you can understand what I am trying to do.

Hope someone can help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script use two variable in for loop

I want to create a shell script to add a user and modify its comment field mentioned in a file. 1. File value:- username comment field value xyz123 xyztesting abc123 abctesting def123 deftesting 2. i am using below loop to create user... (2 Replies)
Discussion started by: Anil
2 Replies

2. UNIX for Beginners Questions & Answers

How can I assign awk's variable to shell script's variable?

I have the following script, and I want to assign the output ($10 and $5) from awk to N and L: grdinfo data.grd | awk '{print $10,$5}'| read N L output from gridinfo data.grd is: data.grd 50 100 41 82 -2796 6944 0.016 0.016 3001 2461. where N and L is suppose to be 3001 and 100. I use... (8 Replies)
Discussion started by: geomarine
8 Replies

3. UNIX for Beginners Questions & Answers

How to grep variable in shell script?

Hi Team, I am trying to grep date in a file but it is not working #!/bin/bash d=`date "+ %Y%m%d %H:%M"` cd /scripts /bin/rm -f test1 cat /var/logs/File.txt.0 |grep $"d" >v.txt instead it is showing the complete file output. kindly suggest how should i grep this variable it works... (7 Replies)
Discussion started by: scriptor
7 Replies

4. Shell Programming and Scripting

Unable to pass shell script variable to awk command in same shell script

I have a shell script (.sh) and I want to pass a parameter value to the awk command but I am getting exception, please assist. diff=$1$2.diff id=$2 new=new_$diff echo "My id is $1" echo "I want to sync for user account $id" ##awk command I am using is as below cat $diff | awk... (2 Replies)
Discussion started by: Ashunayak
2 Replies

5. Shell Programming and Scripting

Does a variable lose its value outside the loop in shell script?

hi, when we assign a variable inside a for loop or while loop in a shell script, does it loses its value after comming out of the loop. i am facing this issue. can anyone help me?? (8 Replies)
Discussion started by: Little
8 Replies

6. Shell Programming and Scripting

Issue with passing variable to Grep in a shell script

Hi, I'm trying to check if methods specified in a class have been added to the corrosponding interface. My code below is giving me the following errors: grep: function: No such file or directory grep: import($zipfile): No such file or directory grep: function: No such file or... (1 Reply)
Discussion started by: racshot65
1 Replies

7. Shell Programming and Scripting

How to give a variable output name in a shell script inside a for loop

Hi all I run my program prog.c in the following way : $ ./prog 1 > output.txt where 1 is a user defined initial value used by the program. But now I want to run it for many a thousand initial values, 1-1000, and store all the outputs in different files. Like $ ./prog 1... (1 Reply)
Discussion started by: alice06
1 Replies

8. Shell Programming and Scripting

assign awk's variable to shell script's variable?

Dear All, we have a command output which looks like : Total 200 queues in 30000 Kbytes and we're going to get "200" and "30000" for further process. currently, i'm using : numA=echo $OUTPUT | awk '{print $2}' numB=echo $OUTPUT | awk '{print $5}' my question is : can I use just one... (4 Replies)
Discussion started by: tiger2000
4 Replies

9. Shell Programming and Scripting

Passing a variable to awk while in a shell for loop

I am a newbie to awk and c programming, however am not a unix newbie. However, I do need help with a kshell script I am writing. It is almost complete, the last step is killing me. Any help would be greatly appreciated. What I am trying to do is cat a text file that has usernames. Then, using... (2 Replies)
Discussion started by: synergy_texas
2 Replies

10. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies
Login or Register to Ask a Question
Mail::Mbox::MessageParser(3pm)				User Contributed Perl Documentation			    Mail::Mbox::MessageParser(3pm)

NAME
Mail::Mbox::MessageParser - A fast and simple mbox folder reader SYNOPSIS
#!/usr/bin/perl use Mail::Mbox::MessageParser; my $file_name = 'mail/saved-mail'; my $file_handle = new FileHandle($file_name); # Set up cache. (Not necessary if enable_cache is false.) Mail::Mbox::MessageParser::SETUP_CACHE( { 'file_name' => '/tmp/cache' } ); my $folder_reader = new Mail::Mbox::MessageParser( { 'file_name' => $file_name, 'file_handle' => $file_handle, 'enable_cache' => 1, 'enable_grep' => 1, } ); die $folder_reader unless ref $folder_reader; # Any newlines or such before the start of the first email my $prologue = $folder_reader->prologue; print $prologue; # This is the main loop. It's executed once for each email while(!$folder_reader->end_of_file()) { my $email = $folder_reader->read_next_email(); print $$email; } DESCRIPTION
This module implements a fast but simple mbox folder reader. One of three implementations (Cache, Grep, Perl) will be used depending on the wishes of the user and the system configuration. The first implementation is a cached-based one which stores email information about mailboxes on the file system. Subsequent accesses will be faster because no analysis of the mailbox will be needed. The second implementation is one based on GNU grep, and is significantly faster than the Perl version for mailboxes which contain very large(10MB) emails. The final implementation is a fast Perl-based one which should always be applicable. The Cache implementation is about 6 times faster than the standard Perl implementation. The Grep implementation is about 4 times faster than the standard Perl implementation. If you have GNU grep, it's best to enable both the Cache and Grep implementations. If the cache information is available, you'll get very fast speeds. Otherwise, you'll take about a 1/3 performance hit when the Grep version is used instead. The overriding requirement for this module is speed. If you wish more sophisticated parsing, use Mail::MboxParser (which is based on this module) or Mail::Box. METHODS AND FUNCTIONS SETUP_CACHE(...) SETUP_CACHE( { 'file_name' => <cache file name> } ); <cache file name> - the file name of the cache Call this function once to set up the cache before creating any parsers. You must provide the location to the cache file. There is no default value. new(...) new( { 'file_name' => <mailbox file name>, 'file_handle' => <mailbox file handle>, 'enable_cache' => <1 or 0>, 'enable_grep' => <1 or 0>, 'force_processing' => <1 or 0>, 'debug' => <1 or 0>, } ); <mailbox file name> - the file name of the mailbox <mailbox file handle> - the already opened file handle for the mailbox <enable_cache> - true to attempt to use the cache implementation <enable_grep> - true to attempt to use the grep implementation <force_processing> - true to force processing of files that look invalid <debug> - true to print some debugging information to STDERR The constructor takes either a file name or a file handle, or both. If the file handle is not defined, Mail::Mbox::MessageParser will attempt to open the file using the file name. You should always pass the file name if you have it, so that the parser can cache the mailbox information. This module will automatically decompress the mailbox as necessary. If a filename is available but the file handle is undef, the module will call either bzip2, or gzip to decompress the file in memory if the filename ends with .tz, .bz2, or .gz, respectively. If the file handle is defined, it will detect the type of compression and apply the correct decompression program. The Cache, Grep, or Perl implementation of the parser will be loaded, whichever is most appropriate. For example, the first time you use caching, there will be no cache. In this case, the grep implementation can be used instead. The cache will be updated in memory as the grep implementation parses the mailbox, and the cache will be written after the program exits. The file name is optional, in which case enable_cache and enable_grep must both be false. force_processing will cause the module to process folders that look to be binary, or whose text data doesn't look like a mailbox. Returns a reference to a Mail::Mbox::MessageParser object on success, and a scalar desribing an error on failure. ("Not a mailbox", "Can't open <filename>: <system error>", "Can't execute <uncompress command> for file <filename>" reset() Reset the filehandle and all internal state. Note that this will not work with filehandles which are streams. If there is enough demand, I may add the ability to store the previously read stream data internally so that reset() will work correctly. endline() Returns " " or " ", depending on the file format. prologue() Returns any newlines or other content at the start of the mailbox prior to the first email. end_of_file() Returns true if the end of the file has been encountered. line_number() Returns the line number for the start of the last email read. number() Returns the number of the last email read. (i.e. The first email will have a number of 1.) length() Returns the length of the last email read. offset() Returns the byte offset of the last email read. read_next_email() Returns a reference to a scalar holding the text of the next email in the mailbox, or undef at the end of the file. BUGS
No known bugs. Contact david@coppit.org for bug reports and suggestions. AUTHOR
David Coppit <david@coppit.org>. LICENSE
This software is distributed under the terms of the GPL. See the file "LICENSE" for more information. HISTORY
This code was originally part of the grepmail distribution. See http://grepmail.sf.net/ for previous versions of grepmail which included early versions of this code. SEE ALSO
Mail::MboxParser, Mail::Box perl v5.10.1 2010-06-06 Mail::Mbox::MessageParser(3pm)