Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks
Top Forums Shell Programming and Scripting Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks Post 302766025 by RudiC on Monday 4th of February 2013 04:37:23 AM
Old 02-04-2013
You have a data inconsistency in your sample file: the header line has six fields, while the data rows have seven. I had to eliminate that in order to have following work. Try
Code:
awk '    {for (i=1; i<=NF; i++)                            # for every field in every line
           {LG[i] = LG[i] * ($i=="") + (NR==1)             # test if $i is empty, except for line 1
            Ar[NR, i] = $i                                 # save field by line and fld No.
            }
         }
     END {for (j=1; j<=NR; j++)                            # go through all lines of entire file
           {for (i=1; i<=NF; i++)                          # with all saved fields
              if (!LG[i]) rec = rec (rec?OFS:z) Ar[j,i]    # compose output record unless field was empty in all lines
            printf "%s\n", rec; rec = ""                   # print it
           }
         }
    ' FS=, OFS=, file
Key, Data1,Data2,Data3,Data4
A,5,6,,10
A,3,4,,3
B,1,,4,5
B,2,,3,4

Last edited by RudiC; 02-05-2013 at 04:42 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

deleting rows & columns form a csv file

Hi , I want to delete some rows & columns from file. can someone please help me on this? Regards. (2 Replies)
Discussion started by: code19
2 Replies

2. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

3. Shell Programming and Scripting

deleting rows that dont have a certain # of columns

Hi, I want to delete rows that dont have a certain # of columns. In my case, rows that are less than 8 should be removed (those greater than 8 are ok). For instance: 1 2 3 4 5 6 7 8 2 3 2 4 3 2 1 5 1 2 3 4 5 6 8 2 2 4 3 1 1 1 1 1 1 1 1 1 after: 1... (8 Replies)
Discussion started by: gisele_l
8 Replies

4. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

5. UNIX for Dummies Questions & Answers

Deleting all rows with empty columns

I have a text file that looks like this: 1 rs523634 8.22486 1 1 rs585160 8.22488 1 rs497228 8.2249 1 1 rs600933 8.225 1 rs480106 8.22531 1 rs600199 8.22533 1 rs529015 8.22534 1 rs598894 8.22534 I want to delete the rows with empty... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

7. Shell Programming and Scripting

Converting rows to columns in csv file

Hi, I have a requirement to convert rows into columns. data looks like: c1,c2,c3,.. r1,r2,r3,.. p1,p2,p3,.. and so on.. output shud be like this: c1,r1,p1,.. c2,r2,p2,.. c3,r3,p3,.. Thanks in advance, (12 Replies)
Discussion started by: Divya1987
12 Replies

8. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

9. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

LEARN ABOUT SUSE

pgmtoy4m

pgmtoy4m(1)							MJPEG tools manual						       pgmtoy4m(1)

NAME

       pgmtoy4m - Convert mpeg2dec pgm and pgmpipe output to YUV4MPEG2

SYNOPSIS

       pgmtoy4m [options]

DESCRIPTION

       pgmtoy4m  repacks the PGM output from mpeg2dec into YUV4MPEG2 4:2:0p.	No actual changes to the data are made.  The data is unpacked from
       the quasi-PGM format and placed in YUV4MPEG2 format with the specified sample aspect, frame rate and field order.  Output must be either to
       a pipe or a file, leaving stdout as a terminal will elicit an error and the program will exit.

OPTIONS

       pgmtoy4m accepts the following options:

       -v num
	    Set verbosity level.  (default: 0)

       -h   Print a usage summary and exit.

       -i t|b|p
	    Set the interlacing mode.  Top field first (t), bottom field first (b) or progressive/none (p).  (default: t)

       -a sar
	    Set the sample aspect ratio.  (default: 10:11)

       -r frame rate
	    Set the frame rate.  (default: 30000:1001)

EXAMPLES

       mpeg2dec -s -o pgmpipe file.mpg | pgmtoy4m -a 10:11 -r 24000:1001 -i b | yuvplay

       mpeg2dec -s -o pgm file.mpg
       cat *.pgm | pgmtoy4m -r 30000:1001 -i t | yuvplay

NOTES

       This program is specifically designed to work with the output of mpeg2dec and is NOT a general PGM (NetPBM) tool!

       From the introduction in the source file:

	* Note: mpeg2dec uses a variation of the PGM format - they're really not
	* "Grey Maps" but rather a catenation of the 420P data (commonly called
	* "YUV").    The type is P5 ("raw") and the number of rows is really
	* the total of the Y', Cb and Cr heights.   The Cb and Cr data is "joined"
	* together.  After the Y' rows you have 1 row of Cb and 1 row of Cr per
	* "row" of PGM data.

	* NOTE: You MAY need to know the field order (top/bottom field first),
	*      sample aspect ratio and frame rate because the PGM format makes
	*      none of that information available!

AUTHOR

       This manpage was written by Steven Schultz (sms at 2bsd dot com).

BUGS

       Possible but none known at the present time.

MJPEG Linux Square						 16 December 2003						       pgmtoy4m(1)
All times are GMT -4. The time now is 01:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy