Sponsored Content
Top Forums Shell Programming and Scripting Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks Post 302766025 by RudiC on Monday 4th of February 2013 04:37:23 AM
Old 02-04-2013
You have a data inconsistency in your sample file: the header line has six fields, while the data rows have seven. I had to eliminate that in order to have following work. Try
Code:
awk '    {for (i=1; i<=NF; i++)                            # for every field in every line
           {LG[i] = LG[i] * ($i=="") + (NR==1)             # test if $i is empty, except for line 1
            Ar[NR, i] = $i                                 # save field by line and fld No.
            }
         }
     END {for (j=1; j<=NR; j++)                            # go through all lines of entire file
           {for (i=1; i<=NF; i++)                          # with all saved fields
              if (!LG[i]) rec = rec (rec?OFS:z) Ar[j,i]    # compose output record unless field was empty in all lines
            printf "%s\n", rec; rec = ""                   # print it
           }
         }
    ' FS=, OFS=, file
Key, Data1,Data2,Data3,Data4
A,5,6,,10
A,3,4,,3
B,1,,4,5
B,2,,3,4


Last edited by RudiC; 02-05-2013 at 04:42 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

deleting rows & columns form a csv file

Hi , I want to delete some rows & columns from file. can someone please help me on this? Regards. (2 Replies)
Discussion started by: code19
2 Replies

2. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

3. Shell Programming and Scripting

deleting rows that dont have a certain # of columns

Hi, I want to delete rows that dont have a certain # of columns. In my case, rows that are less than 8 should be removed (those greater than 8 are ok). For instance: 1 2 3 4 5 6 7 8 2 3 2 4 3 2 1 5 1 2 3 4 5 6 8 2 2 4 3 1 1 1 1 1 1 1 1 1 after: 1... (8 Replies)
Discussion started by: gisele_l
8 Replies

4. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

5. UNIX for Dummies Questions & Answers

Deleting all rows with empty columns

I have a text file that looks like this: 1 rs523634 8.22486 1 1 rs585160 8.22488 1 rs497228 8.2249 1 1 rs600933 8.225 1 rs480106 8.22531 1 rs600199 8.22533 1 rs529015 8.22534 1 rs598894 8.22534 I want to delete the rows with empty... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

7. Shell Programming and Scripting

Converting rows to columns in csv file

Hi, I have a requirement to convert rows into columns. data looks like: c1,c2,c3,.. r1,r2,r3,.. p1,p2,p3,.. and so on.. output shud be like this: c1,r1,p1,.. c2,r2,p2,.. c3,r3,p3,.. Thanks in advance, (12 Replies)
Discussion started by: Divya1987
12 Replies

8. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

9. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies
TIFFReadScanline(3TIFF) 												   TIFFReadScanline(3TIFF)

NAME
TIFFReadScanline - read and decode a scanline of data from an open TIFF file SYNOPSIS
#include <tiffio.h> int TIFFReadScanline(TIFF *tif, tdata_t buf, uint32 row, tsample_t sample) DESCRIPTION
Read the data for the specified row into the (user supplied) data buffer buf. The data are returned decompressed and, in the native byte- and bit-ordering, but are otherwise packed (see further below). The buffer must be large enough to hold an entire scanline of data. Appli- cations should call the routine TIFFScanlineSize to find out the size (in bytes) of a scanline buffer. The row parameter is always used by TIFFReadScanline; the sample parameter is used only if data are organized in separate planes (PlanarConfiguration=2). NOTES
The library attempts to hide bit- and byte-ordering differences between the image and the native machine by converting data to the native machine order. Bit reversal is done if the FillOrder tag is opposite to the native machine bit order. 16- and 32-bit samples are automati- cally byte-swapped if the file was written with a byte order opposite to the native machine byte order, In C++ the sample parameter defaults to 0. RETURN VALUES
TIFFReadScanline returns -1 if it detects an error; otherwise 1 is returned. DIAGNOSTICS
All error messages are directed to the TIFFError(3TIFF) routine. Compression algorithm does not support random access. Data was requested in a non-sequential order from a file that uses a compression algorithm and that has RowsPerStrip greater than one. That is, data in the image is stored in a compressed form, and with multiple rows packed into a strip. In this case, the library does not support random access to the data. The data should either be accessed sequentially, or the file should be converted so that each strip is made up of one row of data. BUGS
Reading subsampled YCbCR data does not work correctly because, for PlanarConfiguration=2 the size of a scanline is not calculated on a per- sample basis, and for PlanarConfiguration=1 the library does not unpack the block-interleaved samples; use the strip- and tile-based inter- faces to read these formats. SEE ALSO
TIFFOpen(3TIFF), TIFFReadEncodedStrip(3TIFF), TIFFReadRawStrip(3TIFF), libtiff(3TIFF) Libtiff library home page: http://www.remotesensing.org/libtiff/ libtiff October 15, 1995 TIFFReadScanline(3TIFF)
All times are GMT -4. The time now is 03:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy