Sponsored Content
Top Forums Shell Programming and Scripting Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks Post 302766025 by RudiC on Monday 4th of February 2013 04:37:23 AM
Old 02-04-2013
You have a data inconsistency in your sample file: the header line has six fields, while the data rows have seven. I had to eliminate that in order to have following work. Try
Code:
awk '    {for (i=1; i<=NF; i++)                            # for every field in every line
           {LG[i] = LG[i] * ($i=="") + (NR==1)             # test if $i is empty, except for line 1
            Ar[NR, i] = $i                                 # save field by line and fld No.
            }
         }
     END {for (j=1; j<=NR; j++)                            # go through all lines of entire file
           {for (i=1; i<=NF; i++)                          # with all saved fields
              if (!LG[i]) rec = rec (rec?OFS:z) Ar[j,i]    # compose output record unless field was empty in all lines
            printf "%s\n", rec; rec = ""                   # print it
           }
         }
    ' FS=, OFS=, file
Key, Data1,Data2,Data3,Data4
A,5,6,,10
A,3,4,,3
B,1,,4,5
B,2,,3,4


Last edited by RudiC; 02-05-2013 at 04:42 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

deleting rows & columns form a csv file

Hi , I want to delete some rows & columns from file. can someone please help me on this? Regards. (2 Replies)
Discussion started by: code19
2 Replies

2. Shell Programming and Scripting

Deleting columns from CSV file

Hi All, Am working on perl script which should delete columns in existing CSV file. If my file is : AA,BB,CC,DD 00,11,22,33 00,55,66,77 00,99,88,21 AA,BB... are all my headers can come in any order (e.g AA,CC,BB...) and rest are values. I want to delete column CC... Can anybody help... (2 Replies)
Discussion started by: darshakraut
2 Replies

3. Shell Programming and Scripting

deleting rows that dont have a certain # of columns

Hi, I want to delete rows that dont have a certain # of columns. In my case, rows that are less than 8 should be removed (those greater than 8 are ok). For instance: 1 2 3 4 5 6 7 8 2 3 2 4 3 2 1 5 1 2 3 4 5 6 8 2 2 4 3 1 1 1 1 1 1 1 1 1 after: 1... (8 Replies)
Discussion started by: gisele_l
8 Replies

4. UNIX for Advanced & Expert Users

Unix Bash: substitute columns in .csv using other .csv columns

Hi All, I have two .csv's input.csv having values as (7 columns) ABC,A19907103,ABC DEV YUNG,2.17,1000,2157,07/07/2006 XYZ,H00213850,MM TRUP HILL,38.38,580,23308,31/08/2010 output.csv having (25 columns) A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y... (4 Replies)
Discussion started by: abhivyas
4 Replies

5. UNIX for Dummies Questions & Answers

Deleting all rows with empty columns

I have a text file that looks like this: 1 rs523634 8.22486 1 1 rs585160 8.22488 1 rs497228 8.2249 1 1 rs600933 8.225 1 rs480106 8.22531 1 rs600199 8.22533 1 rs529015 8.22534 1 rs598894 8.22534 I want to delete the rows with empty... (2 Replies)
Discussion started by: evelibertine
2 Replies

6. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

7. Shell Programming and Scripting

Converting rows to columns in csv file

Hi, I have a requirement to convert rows into columns. data looks like: c1,c2,c3,.. r1,r2,r3,.. p1,p2,p3,.. and so on.. output shud be like this: c1,r1,p1,.. c2,r2,p2,.. c3,r3,p3,.. Thanks in advance, (12 Replies)
Discussion started by: Divya1987
12 Replies

8. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

9. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies
TIFFWriteScanline(3TIFF)												  TIFFWriteScanline(3TIFF)

NAME
TIFFWriteScanline - write a scanline to an open TIFF file SYNOPSIS
#include <tiffio.h> int TIFFWriteScanline(TIFF *tif, tdata_t buf, uint32 row, tsample_t sample) DESCRIPTION
Write data to a file at the specified row. The sample parameter is used only if data are organized in separate planes (PlanarConfigura- tion=2). The data are assumed to be uncompressed and in the native bit- and byte-order of the host machine. The data written to the file is compressed according to the compression scheme of the current TIFF directory (see further below). If the current scanline is past the end of the current subfile, the ImageLength field is automatically increased to include the scanline (except for PlanarConfiguration=2, where the ImageLength cannot be changed once the first data are written). If the ImageLength is increased, the StripOffsets and StripByte- Counts fields are similarly enlarged to reflect data written past the previous end of image. NOTES
The library writes encoded data using the native machine byte order. Correctly implemented TIFF readers are expected to do any necessary byte-swapping to correctly process image data with BitsPerSample greater than 8. The library attempts to hide bit-ordering differences between the image and the native machine by converting data from the native machine order. In C++ the sample parameter defaults to 0. Once data are written to a file for the current directory, the values of certain tags may not be altered; see TIFFSetField(3TIFF) for more information. It is not possible to write scanlines to a file that uses a tiled organization. The routine TIFFIsTiled can be used to determine if the file is organized as tiles or strips. RETURN VALUES
TIFFWriteScanline returns -1 if it immediately detects an error and 1 for a successful write. DIAGNOSTICS
All error messages are directed to the TIFFError(3TIFF) routine. %s: File not open for writing . The file was opened for reading, not writing. Can not write scanlines to a tiled image. An attempt was made to write a scanline to a tiled image. The image is assumed to be organized in tiles because the TileWidth and TileLength tags have been set with TIFFSetField(3TIFF). Compression algorithm does not support random access. Data was written in a non-sequential order to a file that uses a compression algo- rithm and that has RowsPerStrip greater than one. That is, data in the image is to be stored in a compressed form, and with multiple rows packed into a strip. In this case, the library does not support random access to the data. The data should either be written as entire strips, sequentially by rows, or the value of RowsPerStrip should be set to one. %s: Must set "ImageWidth" before writing data. The image's width has not be set before the first write. See TIFFSetField(3TIFF) for information on how to do this. %s: Must set "PlanarConfiguration" before writing data. The organization of data has not be defined before the first write. See TIFFSet- Field(3TIFF) for information on how to do this. Can not change "ImageLength" when using separate planes. Separate image planes are being used (PlanarConfiguration=2), but the number of rows has not been specified before the first write. The library supports the dynamic growth of an image only when data are organized in a contiguous manner (PlanarConfiguration=1). %d: Sample out of range, max %d. The sample parameter was greater than the value of the SamplesPerPixel tag. %s: No space for strip arrays . There was not enough space for the arrays that hold strip offsets and byte counts. BUGS
Writing subsampled YCbCR data does not work correctly because, for PlanarConfiguration=2 the size of a scanline is not calculated on a per- sample basis, and for PlanarConfiguration=1 the library does not pack the block-interleaved samples. SEE ALSO
TIFFOpen(3TIFF), TIFFWriteEncodedStrip(3TIFF), TIFFWriteRawStrip(3TIFF), libtiff(3TIFF) Libtiff library home page: http://www.remotesensing.org/libtiff/ libtiff December 16, 1991 TIFFWriteScanline(3TIFF)
All times are GMT -4. The time now is 05:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy