Sponsored Content
Top Forums Shell Programming and Scripting awk processing of variable number of fields data file Post 302874613 by radudownload on Friday 15th of November 2013 06:49:05 PM
Old 11-15-2013
Thank you guys!

I've only implemented Scrutinizers suggestion, and it works well for me.

For CarloM: (I'm not an awk expert, but ...) can you please tell me where should I state in your second code what file should be processed? Thank you!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this: sec;src;dst;proto 421;10.10.10.1;10.10.10.2;tcp 426;10.10.10.3;10.10.10.4;udp 442;10.10.10.5;10.10.10.6;tcp sec;src;fac;dst;proto 521;10.10.10.1;ab;10.10.10.2;tcp... (3 Replies)
Discussion started by: eric4
3 Replies

2. Shell Programming and Scripting

Parse apart strings of comma separated data with varying number of fields

I have a situation where I am reading a text file line-by-line. Those lines of data contain comma separated fields of data. However, each line can vary in the number of fields it can contain. What I need to do is parse apart each line and write each field of data found (left to right) into a file.... (7 Replies)
Discussion started by: 2reperry
7 Replies

3. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,... (4 Replies)
Discussion started by: axo959
4 Replies

4. Shell Programming and Scripting

Help with data processing, maybe awk

I have a file, first 5 columns are very normal, like "1107",106027,71400,"Y","BIOLOGY",, however, the 6th columns, the user can put comments, anything, just any characters, like new line, double quote, single quote, whatever from the keyboard, like"Please load my previous SOM597G course content in... (3 Replies)
Discussion started by: freelong
3 Replies

5. Shell Programming and Scripting

number of fields in a text file as a variable - perl

I am looking for perl code to get following o/p. If a line has more than 7 fields then value in field 7 onwards is BHA_GRP1, BHA_GRP2, BHA_GRP3, BHA_GRP4 etc. Here is example of what I am trying to achieve. INPUT File: VAH NIC_TYPE CONFIG SIZE_GB PILO KOM BHA_GRP1 BHA_GRP2 BHA_GRP3...... 2... (1 Reply)
Discussion started by: dynamax
1 Replies

6. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

7. Shell Programming and Scripting

Data processing using awk

Hello, I have some bitrate data in a csv which is in an odd format and is difficult to process in Excel when I have thousands of rows. Therefore, I was thinking of doing this in bash and using awk as the primary application except that due to its complication, I'm a little stuck. ... (24 Replies)
Discussion started by: shadyuk
24 Replies

8. Shell Programming and Scripting

[ksh88 and awk] Number of fields with a value.

Hi, With: # VALUES="one~two~~~" # echo $VALUES | awk 'BEGIN {FS="~"} {print NF}' 5 I can determine the number of fields. How to determine the number of fields with a value ? In this case 2. Thanks in advance, ejdv (6 Replies)
Discussion started by: ejdv
6 Replies

9. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

10. Shell Programming and Scripting

Setting the number of fields using awk

Hi Guys, I've obviously had a senior moment here, what I'm trying to do is set the number of fields to 35 in a csv these should be appended to the end of the line. But what I'm getting is:- Source Data $ head out_file_01.txt N1000,024,2809003,,,3,DYNAMIC AVLEASE INC,PO BOX... (10 Replies)
Discussion started by: gull04
10 Replies
STATES(1)							      STATES								 STATES(1)

NAME
states - awk alike text processing tool SYNOPSIS
states [-hV] [-D var=val] [-f file] [-o outputfile] [-s startstate] [-W level] [filename ...] DESCRIPTION
States is an awk-alike text processing tool with some state machine extensions. It is designed for program source code highlighting and to similar tasks where state information helps input processing. At a single point of time, States is in one state, each quite similar to awk's work environment, they have regular expressions which are matched from the input and actions which are executed when a match is found. From the action blocks, states can perform state transitions; it can move to another state from which the processing is continued. State transitions are recorded so states can return to the calling state once the current state has finished. The biggest difference between states and awk, besides state machine extensions, is that states is not line-oriented. It matches regular expression tokens from the input and once a match is processed, it continues processing from the current position, not from the beginning of the next input line. OPTIONS
-D var=val, --define=var=val Define variable var to have string value val. Command line definitions overwrite variable definitions found from the config file. -f file, --file=file Read state definitions from file file. As a default, states tries to read state definitions from file states.st in the current working directory. -h, --help Print short help message and exit. -o file, --output=file Save output to file file instead of printing it to stdout. -s state, --state=state Start execution from state state. This definition overwrites start state resolved from the start block. -V, --version Print states version and exit. -W level, --warning=level Set the warning level to level. Possible values for level are: light light warnings (default) all all warnings STATES PROGRAM FILES
States program files can contain on start block, startrules and namerules blocks to specify the initial state, state definitions and expressions. The start block is the main() of the states program, it is executed on script startup for each input file and it can perform any initial- ization the script needs. It normally also calls the check_startrules() and check_namerules() primitives which resolve the initial state from the input file name or the data found from the begining of the input file. Here is a sample start block which initializes two vari- ables and does the standard start state resolving: start { a = 1; msg = "Hello, world!"; check_startrules (); check_namerules (); } Once the start block is processed, the input processing is continued from the initial state. The initial state is resolved by the information found from the startrules and namerules blocks. Both blocks contain regular expression - symbol pairs, when the regular expression is matched from the name of from the beginning of the input file, the initial state is named by the corresponding symbol. For example, the following start and name rules can distinguish C and Fortran files: namerules { /.(c|h)$/ c; /.[fF]$/ fortran; } startrules { /- [cC] -/ c; /- fortran -/ fortran; } If these rules are used with the previously shown start block, states first check the beginning of input file. If it has string -*- c -*-, the file is assumed to contain C code and the processing is started from state called c. If the beginning of the input file has string -*- fortran -*-, the initial state is fortran. If none of the start rules matched, the name of the input file is matched with the namerules. If the name ends to suffix c or C, we go to state c. If the suffix is f or F, the initial state is fortran. If both start and name rules failed to resolve the start state, states just copies its input to output unmodified. The start state can also be specified from the command line with option -s, --state. State definitions have the following syntax: state { expr {statements} ... } where expr is: a regular expression, special expression or symbol and statements is a list of statements. When the expression expr is matched from the input, the statement block is executed. The statement block can call states' primitives, user-defined subroutines, call other states, etc. Once the block is executed, the input processing is continued from the current intput position (which might have been changed if the statement block called other states). Special expressions BEGIN and END can be used in the place of expr. Expression BEGIN matches the beginning of the state, its block is called when the state is entered. Expression END matches the end of the state, its block is executed when states leaves the state. If expr is a symbol, its value is looked up from the global environment and if it is a regular expression, it is matched to the input, oth- erwise that rule is ignored. The states program file can also have top-level expressions, they are evaluated after the program file is parsed but before any input files are processed or the start block is evaluated. PRIMITIVE FUNCTIONS
call (symbol) Move to state symbol and continue input file processing from that state. Function returns whatever the symbol state's terminating return statement returned. check_namerules () Try to resolve start state from namerules rules. Function returns 1 if start state was resolved or 0 otherwise. check_startrules () Try to resolve start state from startrules rules. Function returns 1 if start state was resolved or 0 otherwise. concat (str, ...) Concanate argument strings and return result as a new string. float (any) Convert argument to a floating point number. getenv (str) Get value of environment variable str. Returns an empty string if variable var is undefined. int (any) Convert argument to an integer number. length (item, ...) Count the length of argument strings or lists. list (any, ...) Create a new list which contains items any, ... panic (any, ...) Report a non-recoverable error and exit with status 1. Function never returns. print (any, ...) Convert arguments to strings and print them to the output. range (source, start, end) Return a sub-range of source starting from position start (inclusively) to end (exclusively). Argument source can be string or list. regexp (string) Convert string string to a new regular expression. regexp_syntax (char, syntax) Modify regular expression character syntaxes by assigning new syntax syntax for character char. Possible values for syntax are: 'w' character is a word constituent ' ' character isn't a word constituent regmatch (string, regexp) Check if string string matches regular expression regexp. Functions returns a boolean success status and sets sub-expression reg- isters $n. regsub (string, regexp, subst) Search regular expression regexp from string string and replace the matching substring with string subst. Returns the resulting string. The substitution string subst can contain $n references to the n:th parenthesized sup-expression. regsuball (string, regexp, subst) Like regsub but replace all matches of regular expression regexp from string string with string subst. split (regexp, string) Split string string to list considering matches of regular rexpression regexp as item separator. sprintf (fmt, ...) Format arguments according to fmt and return result as a string. strcmp (str1, str2) Perform a case-sensitive comparision for strings str1 and str2. Function returns a value that is: -1 string str1 is less than str2 0 strings are equal 1 string str1 is greater than str2 string (any) Convert argument to string. strncmp (str1, str2, num) Perform a case-sensitive comparision for strings str1 and str2 comparing at maximum num characters. substring (str, start, end) Return a substring of string str starting from position start (inclusively) to end (exclusively). BUILTIN VARIABLES
$. current input line number $n the nth parenthesized regular expression sub-expression from the latest state regular expression or from the regmatch primitive $` everything before the matched regular rexpression. This is usable when used with the regmatch primitive; the contents of this variable is undefined when used in action blocks to refer the data before the block's regular expression. $B an alias for $` argv list of input file names filename name of the current input file program name of the program (usually states) version program version string FILES
/usr/share/enscript/enscript.st enscript's states definitions SEE ALSO
awk(1), enscript(1) AUTHOR
Markku Rossi <mtr@iki.fi> <http://www.iki.fi/~mtr/> GNU Enscript WWW home page: <http://www.iki.fi/~mtr/genscript/> STATES
Jun 6, 1997 STATES(1)
All times are GMT -4. The time now is 12:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy