Sponsored Content
Top Forums Shell Programming and Scripting Convert comma separated file to fix length Post 302814309 by apenkov on Wednesday 29th of May 2013 09:33:46 AM
Old 05-29-2013
Convert comma separated file to fix length

Hi, I am converting a comma separated file to fixed field lenght and I am using that:
Code:
COLUMNS="25 24 67 26 39 63 20 34 35 14 397"
(
cat $indir/input_file.dat | \
$AWK -v columns="$COLUMNS" '
  BEGIN {
    FS=",";
    OFS="";
    split(columns, arr, " ");
  }
  {
    for(i=1; i<=NF; i++)
      printf("%-*s%c", arr[i], $i, (i==NF) ? RS : OFS)
  }
') >> $outdir/output_file.dat

It is working fine, but most of the files are big and the performance is very slow. Any ideas how can I make it faster?

Thanks!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

2. Shell Programming and Scripting

How to format file into comma separated field

Guys, Need you help, i have a a file content that look like this. Nokia 3330 <spaces><spaces><more spaces>+76451883874 Nokia 3610 +87467361615 so on and so forth, - there are so many spaces in between. - e.g.... (5 Replies)
Discussion started by: shtobias
5 Replies

3. Shell Programming and Scripting

parsing comma separated file

Hi, I have a file with th elist of patches separated by comma, like below: patch1, patch 2, patch 3................ t\The number of patches is not known as it changes every time. I need assistance in writing a routine such as it will take patch1 as first variable and performs the... (4 Replies)
Discussion started by: avikaljain
4 Replies

4. Shell Programming and Scripting

Comma separated file

Hi all, I have the following files types: FileA: 100, 23, 33, FileB: 22, 45, 78, and i want to make File C: 100,22 23,45 33,78 any nice suggestions for making it easy. (3 Replies)
Discussion started by: hen1610
3 Replies

5. Shell Programming and Scripting

Need Help - comma inside double quote in comma separated csv,

Hello there, I have a comma separated csv , and all the text field is wrapped by double quote. Issue is some text field contain comma as well inside double quote. so it is difficult to process. Input in the csv file is , 1,234,"abc,12,gh","GH234TY",34 I need output like below,... (8 Replies)
Discussion started by: Uttam Maji
8 Replies

6. Shell Programming and Scripting

How to split the comma separated file?

Hi, I have a filein unix like ABC,CDE BCD,KHL and the output i need is like column1 column2 ABC,CDE ABC ABC,CDE CDE BCD,KHL BCD BCD,KHL KHL. Can some body help me out? Hi, The code is working fine. But in my file each row does not have always 1 comma. It may... (6 Replies)
Discussion started by: jagdishrout
6 Replies

7. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

8. Shell Programming and Scripting

Convert column to quote and comma separated row

Hi, I have a list of tables in a file.txt C_CLAIM C_HLD C_PROVIDER I want the output to be 'C_CLAIM','C_HLD','C_PROVIDER' Currently I'm usin awk and getting output which is almost correct but still has minor defects awk -vORS="','" '{ print $1 }' file.txt The output of... (4 Replies)
Discussion started by: wahi80
4 Replies

9. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

10. Shell Programming and Scripting

Convert fixed value fields to comma separated values

Hi All, Hope you are doing Great!!!. Today i have came up with a problem to say exactly it was for performance improvement. I have written code in perl as a solution for this to cut in specific range, but it is taking time to run for files thousands of lines so i am expecting a sed... (9 Replies)
Discussion started by: mad man
9 Replies
COLUMN(1)						    BSD General Commands Manual 						 COLUMN(1)

NAME
column -- columnate lists SYNOPSIS
column [-tx] [-c columns] [-s sep] [file ...] DESCRIPTION
The column utility formats its input into multiple columns. Rows are filled before columns. Input is taken from file operands, or, by default, from the standard input. Empty lines are ignored. The options are as follows: -c Output is formatted for a display columns wide. -s Specify a set of characters to be used to delimit columns for the -t option. -t Determine the number of columns the input contains and create a table. Columns are delimited with whitespace, by default, or with the characters supplied using the -s option. Useful for pretty-printing displays. -x Fill columns before filling rows. Column exits 0 on success, >0 if an error occurred. ENVIRONMENT
COLUMNS The environment variable COLUMNS is used to determine the size of the screen if no other information is available. EXAMPLES
(printf "PERM LINKS OWNER GROUP SIZE MONTH DAY HH:MM/YEAR NAME " ; ls -l | sed 1d) | column -t SEE ALSO
colrm(1), ls(1), paste(1), sort(1) HISTORY
The column command appeared in 4.3BSD-Reno. BSD
June 6, 1993 BSD
All times are GMT -4. The time now is 07:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy