Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Insert blanks for missing fields and reformat to csv
Top Forums Shell Programming and Scripting Insert blanks for missing fields and reformat to csv Post 302345095 by durden_tyler on Tuesday 18th of August 2009 11:48:59 AM
Old 08-18-2009
One way of doing it with Perl:

Code:
perl -lne 'chomp; /\|/ && push @x,$_;
           if(/^$/ && defined @x){print join ",",@x; @x=()}
           END{print join ",",@x}' file

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

CSV to SQL insert: Awk for strings with multiple lines in csv

Hi Fellows, I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem: I have csv files in the following pipe-delimited format: Column1|Column2|Column3|Column4|NEWLINE Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies

2. Shell Programming and Scripting

Insert missing field using perl,sed,awk

sample file (comma as field separators) MessageFlow,1,BusIntBatchMgr,a OOBEvent,1,BusIntBatchMgr,a TaskEvents,1,,a MTTrace,1,,a MTWarning,,1,a MessageFlow,1,Batch,a OOBEvent,1,Batch,a TaskEvents,1,,a EAISAPIdocWizard,1,BusIntMgr,a EAISAPBAPIWizard,1,BusIntMgr,a... (3 Replies)
Discussion started by: vrclm
3 Replies

3. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

4. UNIX for Dummies Questions & Answers

How to combine and insert missing consecutive numbers - awk or script?

Hi all, I have two (2) sets of files that are based on some snapshots of database that I want to merge and insert any missing sequential number. Below are example representation of these files: file1: DATE TIME COL1 COL2 COL3 COL4 ID 01/10/2013 0800 100 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

5. Shell Programming and Scripting

Reformat csv file

Hi, I have a csv file with content like: 1,0,100 1,1,150 2,0,200 2,1,250 3,0,300 3,1,350 I want an output such that all numbers in 3rd col where 2nd col is "0" come in the same col in the output. The same goes for numbers where 2nd col is "1". 1 100 150 2 200 250 3 300 350 Tnx... (2 Replies)
Discussion started by: jamaje
2 Replies

6. Shell Programming and Scripting

Insert missing values

Hi, please help with this, I need to insert missing values into a matrix for a regression analysis. I have made up an example. The first three columns are variables with levels and the next 3 are values, the 4th column missing values should be replaced by 0s, and 5th and 6th column missing... (3 Replies)
Discussion started by: ritakadm
3 Replies

7. Shell Programming and Scripting

Compare 2 files and find missing fields awk

Hello experts! I have 2 files. file1 is a list file containing uniquely names. e.g.: name1 number number name2 number number name5 number number name10 number number ... file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g. name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies

8. Shell Programming and Scripting

How to print the missing fields outside the for loop in Korn shell?

I have 2 for loop in my program , first one will list files based on timestamp and second one list the files based on type(RPT / SUB_RPT).Here is my code: #!/bin/ksh STG_DIR=/home/stg for pattern in `find $STG_DIR -type f -name 'IBC*csv' | awk -F'' '{print $(NF-1)}' | sort -u` do echo... (2 Replies)
Discussion started by: ann15
2 Replies

9. Shell Programming and Scripting

awk to insert missing string based on pattern in file

Using the file below, which will always have the first indicated by the digit after the - and last id in it, indicated by the digit after the -, I am trying to use awk to print the missing line or lines in file following the pattern of the previous line. For example, in the file below the next... (4 Replies)
Discussion started by: cmccabe
4 Replies

10. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

LEARN ABOUT OSF1

invcutter

invcutter(1)						      General Commands Manual						      invcutter(1)

NAME

       invcutter - Generates a subset inventory file

SYNOPSIS

       /usr/lbin/invcutter [-d] [-f root-path] [-v version-code]

OPTIONS

       Enables debugging.  No useful diagnostics are printed.  Specifies an alternate root path for finding file attribute information.  Specifies
       a 3-digit version code for use in the version field of the output records.  The default version code is 010.

DESCRIPTION

       The contents of a software product kit for the setld(8) command are specified by a master inventory file whose format is described  in  the
       stl_mi(4)  reference  page.   The  invcutter command processes a master inventory file by reading inventory records from standard input and
       writing one record to standard output for each record read.

       The information contained in the output record is derived from the input record and from the file attribute information in the file hierar-
       chy rooted in the current directory.

RESTRICTIONS

       Input records must be sorted in ascending ASCII order on the pathname field.

       The invcutter command cannot process input-specified files which exist as sockets in the file hierarchy.

       If an input-specified file has a link count greater than 1, all other links to the file must be represented in the input.

RETURN VALUES

       An exit status of 0 indicates success.  An exit status of 1 indicates an error.	See ERRORS.

ERRORS

       The program cannot set its working directory to the pathname directory specified with the -f option.  The error-message provides additional
       information.  The nth input record is not in the correct sort order.  All input records must be in ascending ASCII  collating  sequence	on
       the pathname field.  An error has occurred attempting to read the attributes of filename. The error-message explains exactly what happened.
       The file named by pathname is a socket.	Sockets are not supported as valid file types for  distribution.   File  pathname  in  the  master
       inventory  is  linked  to  n  files which do not appear in the master inventory.  Check the inventory for validity with the newinv program.
       This informational message states how many files with unresolved links were detected in the input inventory.

EXAMPLES

       The following command generates inventory records for the master inventory entries in PDS020.mi. Output records contain version fields  set
       to 020: invcutter -v 020 < PDS020.mi

SEE ALSO

       Commands:  kits(1), newinv(1)

       Files:  stl_inv(4), stl_mi(4)

       Guide to Preparing Product Kits

																      invcutter(1)
All times are GMT -4. The time now is 05:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy