Login or Register to Ask a Question and Join Our Community


UNIX and Linux Applications

Initializing columns to NULL an saving data to a new file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications Initializing columns to NULL an saving data to a new file
# 1  
Old 04-07-2010
Initializing columns to NULL an saving data to a new file
Hi

I've been trying to delete columns from a file, and the best option I could come up was to initialise them to NULL.

The reason for this is because I need to compare the differences between 2 files with those specific columns excluded.

Here is my script:

Code:
 
#!/bin/bash
#---------------------------------------------------------------------#This script removes the datetime stamp and pipes the data from the old #file to a new file for comparison
#---------------------------------------------------------------------awk -F; -v oldfile="$1" -v newfile="$2" '{$13=$14=$15=""1}' OFS="," $1 > $2

I parse oldfile1.csv to the script:

Code:
 
User Parameter;Entity Name;Cell ID;Type SubCell / TX / FHSY / DRI / TDMA;Instance CHGR / TX / FHSY / DRI / TDMA;CellR ID;Vendor Parameter;Planned Value;Translated Value;Network Value;Override Level;Override Node;Override Value;Planned Date;Network Date;Vendor;Technology;Version
Access Grant Blocks Reserved;CELL;1A;;;;AGBLK;5;5;1;CELL;1A;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;07B
Access Grant Blocks Reserved;CELL;1B;;;;AGBLK;2;2;1;CELL;1B;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;07B
Access Grant Blocks Reserved;CELL;1C;;;;AGBLK;3;3;1;CELL;1C;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;07B
Access Grant Blocks Reserved;CELL;2A;;;;AGBLK;7;7;1;CELL;2A;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B
Access Grant Blocks Reserved;CELL;2B;;;;AGBLK;4;4;1;CELL;2B;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B
Access Grant Blocks Reserved;CELL;2C;;;;AGBLK;0;0;1;CELL;2C;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B
Access Grant Blocks Reserved;CELL;3A;;;;AGBLK;1;1;1;CELL;3A;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B
Access Grant Blocks Reserved;CELL;3B;;;;AGBLK;6;6;1;CELL;3B;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B
Access Grant Blocks Reserved;CELL;3C;;;;AGBLK;4;4;1;CELL;3C;;4/6/2010 9:01:17 AM;1/12/2010 5:15:15 AM;ERICSSON;GSM;08B

It should initialise the fields with date time stamp to NULL and then parse this new data to another file called newfile1.csv (newfile1.csv currently does not exist)


I run the script:

Code:
 ./export_data_clean.sh oldfile1.csv newfile1.csv

Here is my log results:

Code:
 
$ ./export_data_clean.sh oldfile1.csv newfile1.csv
awk: option requires an argument -- F
Usage: awk [POSIX or GNU style options] -f progfile [--] file ...
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options:          GNU long options:
        -f progfile             --file=progfile
        -F fs                   --field-separator=fs
        -v var=val              --assign=var=val
        -m[fr] val
        -W compat               --compat
        -W copyleft             --copyleft
        -W copyright            --copyright
        -W dump-variables[=file]        --dump-variables[=file]
        -W exec=file            --exec=file
        -W gen-po               --gen-po
        -W help                 --help
        -W lint[=fatal]         --lint[=fatal]
        -W lint-old             --lint-old
        -W non-decimal-data     --non-decimal-data
        -W profile[=file]       --profile[=file]
        -W posix                --posix
        -W re-interval          --re-interval
        -W source=program-text  --source=program-text
        -W traditional          --traditional
        -W usage                --usage
        -W version              --version
To report bugs, see node `Bugs' in `gawk.info', which is
section `Reporting Problems and Bugs' in the printed version.
gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.
Examples:
        gawk '{ sum += $1 }; END { print sum }' file
        gawk -F: '{ print $1 }' /etc/passwd
./export_data_clean.sh: line 6: -v: command not found

I have no idea what is wrong here, can you please assist me.
# 2  
Old 04-07-2010
You have to quote the semicolon in the commandline, like
Code:
awk -F';' ...

# 3  
Old 04-07-2010
I have tried this, and I don't get any errors. However, the newfile is generated as an empty file. How do I parse the data after those rows are initialised to NULL to a new file?
# 4  
Old 04-07-2010
You do not have any output statements in your awk script. This worked for me:
Code:
awk -F';' '{$13=$14=$15="";print;}' OFS="," $1 >$2

# 5  
Old 04-07-2010
Thank you for your help. It works now.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multiple columns replace with null values.

I am trying to replace the partcular columns(Col3,col5,col20,col44,col55,co56,col59,col60,col61,col62,col74,col75,col88,col90,col91,col93,col94,col95) with empty Input file Col1,col2,col3,col4,col5------,col100 1,2,3,4,5,---------,100 3,4,5,6,7,---------,300 Output : ... (3 Replies)
Discussion started by: onesuri
3 Replies

2. Shell Programming and Scripting

awk print columns which are not null

I am working on a file with several columns as below MO_NAME,FAULT_TYPE,CLASS,CODE1,CODE2,CODE3 RXOCF-101,BTS INTERNAL,FAULT CODES CLASS 2A,53,58 RXOCF-101,BTS INTERNAL,FAULT CODES CLASS 2B,24 RXOCF-101,BTS INTERNAL,FAULT CODES CLASS 2A,33 RXOCF-101,BTS INTERNAL,FAULT CODES CLASS 2D,57 ... (12 Replies)
Discussion started by: Rizwan Rasul
12 Replies

3. Shell Programming and Scripting

Check for null values in columns

Hi , I have below data with fixed with of 52 bytes having three columns value data. 01930 MA GLOUCESTER 02033 02025 COHASSET 01960 MA ... (3 Replies)
Discussion started by: sonu_pal
3 Replies

4. Shell Programming and Scripting

AWK- extracting values from columns, saving them and gettins statistics

Hello, I am obviously quite new to unix and awk. I need to parse certain columns of a file (delimited by spaces), and somehow save the value of this column somewhere, together with the value of the column just after it (by pairs; so something like ). I'm then supposed to count the times that... (9 Replies)
Discussion started by: acsg
9 Replies

5. UNIX for Advanced & Expert Users

Create a file and enter data in columns

Hello!!!!!!!! I have an issue regarding inserting records in a file columnwise.Is it possible using awk/nawk script? Example: File1: 1|AAA|25|2 5|qqe|20|7 4|wer|31|81 I need to create a second file in which data can be inserted in a columnwise manner i.e. File2: AAA|25|1|2... (1 Reply)
Discussion started by: abhijeet1409
1 Replies

6. Shell Programming and Scripting

Need Help for Adding Three new columns in existing file from fatching data from file

not required this time (36 Replies)
Discussion started by: Sandeep_Malik
36 Replies

7. Shell Programming and Scripting

Greping columns data from file.

Hi i am using shell script which perform oracle database query and after that output is redirect to some temporary file. the output of this file looks like SQL*Plus: Release 10.2.0.2.0 - Production on Tue Aug 5 16:08:06 2008 Copyright (c) 1982, 2005, Oracle. All Rights Reserved. ... (6 Replies)
Discussion started by: esungoe
6 Replies

8. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies

9. Shell Programming and Scripting

finding null records in data file

I am having a "|" delimited flat file and I have to pick up all the records with the 2nd field having null value. Please suggest. (3 Replies)
Discussion started by: dsravan
3 Replies

10. Shell Programming and Scripting

Help with Data Positioning from Columns in a flat file.

Hi All, I have used this forum many times to solve my many scripting problems. This time, I would like to seek some answers to a problem that I've been head scratching quite a bit on. My Example: I am converting a 2000-byte file into a 300-byte file this file has no delimiters and hardly any... (3 Replies)
Discussion started by: oott1
3 Replies
Login or Register to Ask a Question