Sponsored Content
Top Forums Shell Programming and Scripting awk processing of variable number of fields data file Post 302874473 by CarloM on Friday 15th of November 2013 10:20:41 AM
Old 11-15-2013
You could check explicitly check for every 4th line:
Code:
awk '
NR%4==0 { 4 columns stuff }
NR%4!=0 { 6 columns stuff}'

(EDIT: Or check NF==4/NF==6, as was suggested in a briefly-lived post Smilie)


Or just loop around the number of fields you actually have in each line:
Code:
awk '
{
   for (i=1;i<=NF;i++) {
      printf ("%12.8G ", $i*$i)
   }
   printf "\n"
}'


Last edited by CarloM; 11-15-2013 at 12:00 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this: sec;src;dst;proto 421;10.10.10.1;10.10.10.2;tcp 426;10.10.10.3;10.10.10.4;udp 442;10.10.10.5;10.10.10.6;tcp sec;src;fac;dst;proto 521;10.10.10.1;ab;10.10.10.2;tcp... (3 Replies)
Discussion started by: eric4
3 Replies

2. Shell Programming and Scripting

Parse apart strings of comma separated data with varying number of fields

I have a situation where I am reading a text file line-by-line. Those lines of data contain comma separated fields of data. However, each line can vary in the number of fields it can contain. What I need to do is parse apart each line and write each field of data found (left to right) into a file.... (7 Replies)
Discussion started by: 2reperry
7 Replies

3. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,... (4 Replies)
Discussion started by: axo959
4 Replies

4. Shell Programming and Scripting

Help with data processing, maybe awk

I have a file, first 5 columns are very normal, like "1107",106027,71400,"Y","BIOLOGY",, however, the 6th columns, the user can put comments, anything, just any characters, like new line, double quote, single quote, whatever from the keyboard, like"Please load my previous SOM597G course content in... (3 Replies)
Discussion started by: freelong
3 Replies

5. Shell Programming and Scripting

number of fields in a text file as a variable - perl

I am looking for perl code to get following o/p. If a line has more than 7 fields then value in field 7 onwards is BHA_GRP1, BHA_GRP2, BHA_GRP3, BHA_GRP4 etc. Here is example of what I am trying to achieve. INPUT File: VAH NIC_TYPE CONFIG SIZE_GB PILO KOM BHA_GRP1 BHA_GRP2 BHA_GRP3...... 2... (1 Reply)
Discussion started by: dynamax
1 Replies

6. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

7. Shell Programming and Scripting

Data processing using awk

Hello, I have some bitrate data in a csv which is in an odd format and is difficult to process in Excel when I have thousands of rows. Therefore, I was thinking of doing this in bash and using awk as the primary application except that due to its complication, I'm a little stuck. ... (24 Replies)
Discussion started by: shadyuk
24 Replies

8. Shell Programming and Scripting

[ksh88 and awk] Number of fields with a value.

Hi, With: # VALUES="one~two~~~" # echo $VALUES | awk 'BEGIN {FS="~"} {print NF}' 5 I can determine the number of fields. How to determine the number of fields with a value ? In this case 2. Thanks in advance, ejdv (6 Replies)
Discussion started by: ejdv
6 Replies

9. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

10. Shell Programming and Scripting

Setting the number of fields using awk

Hi Guys, I've obviously had a senior moment here, what I'm trying to do is set the number of fields to 35 in a csv these should be appended to the end of the line. But what I'm getting is:- Source Data $ head out_file_01.txt N1000,024,2809003,,,3,DYNAMIC AVLEASE INC,PO BOX... (10 Replies)
Discussion started by: gull04
10 Replies
SADF(1) 							Linux User's Manual							   SADF(1)

NAME
sadf - Display data collected by sar in multiple formats. SYNOPSIS
sadf [ -C ] [ -d | -j | -p | -x ] [ -H ] [ -h ] [ -T | -t | -U ] [ -V ] [ -P { cpu [,...] | ALL } ] [ -s [ hh:mm:ss ] ] [ -e [ hh:mm:ss ] ] [ -- sar_options ] [ interval [ count ] ] [ datafile ] DESCRIPTION
The sadf command is used for displaying the contents of data files created by the sar(1) command. But unlike sar, sadf can write its data in many different formats (CSV, XML, etc.) The default format is one that can easily be handled by pattern processing commands like awk (see option -p). The sadf command extracts and writes to standard output records saved in the datafile file. This file must have been created by a version of sar which is compatible with that of sadf. If datafile is omitted, sadf uses the standard system activity file, the /var/log/sa/sadd file, where the dd parameter indicates the current day. The interval and count parameters are used to tell sadf to select count records at interval seconds apart. If the count parameter is not set, then all the records saved in the data file will be displayed. All the activity flags of sar may be entered on the command line to indicate which activities are to be reported. Before specifying them, put a pair of dashes (--) on the command line in order not to confuse the flags with those of sadf. Not specifying any flags selects only CPU activity. OPTIONS
-C Tell sadf to display comments present in file. -d Print the contents of the data file in a format that can easily be ingested by a relational database system. The output consists of fields separated by a semicolon. Each record contains the hostname of the host where the file was created, the interval value (or -1 if not applicable), the timestamp in a form easily acceptable by most databases, and additional semicolon separated data fields as specified by sar_options command line options. Note that timestamp output can be controlled by options -T, -t and -U. -e [ hh:mm:ss ] Set the ending time of the report, given in local time. The default ending time is 18:00:00. Hours must be given in 24-hour format. -H Display only the header of the report (when applicable). If no format has been specified, then the header of the data file is dis- played. -h When used in conjunction with option -d, all activities will be displayed horizontally on a single line. -j Print the contents of the data file in JSON (JavaScript Object Notation) format. Timestamps can be controlled by options -T and -t. -P { cpu [,...] | ALL } Tell sadf that processor dependent statistics are to be reported only for the specified processor or processors. Specifying the ALL keyword reports statistics for each individual processor, and globally for all processors. Note that processor 0 is the first pro- cessor. -p Print the contents of the data file in a format that can easily be handled by pattern processing commands like awk. The output con- sists of fields separated by a tab. Each record contains the hostname of the host where the file was created, the interval value (or -1 if not applicable), the timestamp, the device name (or - if not applicable), the field name and its value. Note that timestamp output can be controlled by options -T, -t and -U. -s [ hh:mm:ss ] Set the starting time of the data (given in local time), causing the sadf command to extract records time-tagged at, or following, the time specified. The default starting time is 08:00:00. Hours must be given in 24-hour format. -T Display timestamp in local time instead of UTC (Coordinated Universal Time). -t Display timestamp in the original local time of the data file creator instead of UTC (Coordinated Universal Time). -U Display timestamp (UTC - Coordinated Universal Time) in seconds from the epoch. -V Print version number then exit. -x Print the contents of the data file in XML format. Timestamps can be controlled by options -T and -t. The corresponding DTD (Docu- ment Type Definition) and XML Schema are included in the sysstat source package. They are also available at http://pagesperso- orange.fr/sebastien.godard/download.html ENVIRONMENT
The sadf command takes into account the following environment variable: S_TIME_DEF_TIME If this variable exists and its value is UTC then sadf will use UTC time instead of local time to determine the current daily data file located in the /var/log/sa directory. EXAMPLES
sadf -d /var/log/sa/sa21 -- -r -n DEV Extract memory, swap space and network statistics from system activity file 'sa21', and display them in a format that can be ingested by a database. sadf -p -P 1 Extract CPU statistics for processor 1 (the second processor) from current daily data file, and display them in a format that can easily be handled by a pattern processing command. FILES
/var/log/sa/sadd Indicate the daily data file, where the dd parameter is a number representing the day of the month. AUTHOR
Sebastien Godard (sysstat <at> orange.fr) SEE ALSO
sar(1), sadc(8), sa1(8), sa2(8) http://pagesperso-orange.fr/sebastien.godard/ Linux JULY 2012 SADF(1)
All times are GMT -4. The time now is 03:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy