Sponsored Content
Top Forums Shell Programming and Scripting Delimted to padded conversion with unknown field length Post 302441438 by Michael Stora on Friday 30th of July 2010 05:41:20 PM
Old 07-30-2010
Delimted to padded conversion with unknown field length

I’m looking for an elegant way to convert a delimited file (comma delimited in this case) to padded columns (for printing in non-proportional font) but the length of each column is not known ahead of time. It needs to be calculated for each column from the longest entry in that column in a given file.

I can live with a known number of columns, but I’d rather have something that does not rely on that.

I’d rather not brute force this (with a pre-read and column length arrays) if I’m missing some elegant use of an existing command.

My shell is BASH. This is for a Cygwin in a Windows environment (no choice).

Mike
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with finding length of a field

I have a pipe delimited file. I need to check that the first and second fields are 5 characters long and if not i need to append 0 in front of them to make them 5 characters long. can some body let mwe know how i can find the length of the two fields and then make them 5 characters long if they... (6 Replies)
Discussion started by: dsravan
6 Replies

2. UNIX for Dummies Questions & Answers

Extract substring of unknown length from string

I have a string: hgLogOutput=" +0000 files: forum/web/hook-test.txt /forum/web/hook-test-2.txt description: test" and I want to extract the file names from it, they will always appear between the files: and the description:. I have worked out that I can do this: "$hgLogOutput" | awk '{... (2 Replies)
Discussion started by: klogger
2 Replies

3. UNIX for Dummies Questions & Answers

Conversion problem with date field and formatting.

Hi, My input file contains the record(s) as below with space as FS. 01-01024180 35000 MV010 02/03/09 0306 03060226 03 02-00410330 470000 MV010 02/03/09 0301 03010276 03 1. I need to convert the field 02/03/09 (dd/mm/yy) to yyyymmdd yet retain the Field separator. Using the modified... (2 Replies)
Discussion started by: talk2pawee
2 Replies

4. Shell Programming and Scripting

timestamp field was padded with blanks warning

There are two columns on the data base table, create and update timestamps of datatype timestamp. I dont have these fields on the csv file. So I am doing the below. awk -F , -v d="$(date "+ %Y-%m-%d-%H.%M.%S")" '{ OFS=FS; print $1, $2, $3, $4, d, $5, $6, d }' temp.csv > temp1.csv and then... (3 Replies)
Discussion started by: mitr
3 Replies

5. Shell Programming and Scripting

Adding a field to a file using a conversion table

Hello everyone, Here is what i am trying to accomplish. I have a transaction log that I want to to add a field. The fields in the transaction log are tab delimited FYI. My goal is to add a column specifying the category/type to each item purchased. I have created a two column "conversion table"... (2 Replies)
Discussion started by: SpencerClark
2 Replies

6. Shell Programming and Scripting

Removing characters from end of line (length unknown)

Hi I have a file which contains wrong XML, There are some garbage characters at the end of line that I want to get rid of. Example: <request type="product" ><attributes><pair><name>q</name><value><!]></value></pair><pair><name>start</name><value>1</value></pair></attributes></request>�J ... (7 Replies)
Discussion started by: dirtyd0ggy
7 Replies

7. Shell Programming and Scripting

Flat file-make field length equal to header length

Hello Everyone, I am stuck with one issue while working on abstract flat file which i have to use as input and load data to table. Input Data- ------ ------------------------ ---- ----------------- WFI001 Xxxxxx Control Work Item A Number of Records ------ ------------------------... (5 Replies)
Discussion started by: sonali.s.more
5 Replies

8. Shell Programming and Scripting

Replace a field with a character as per the field length

Hi all, I have a requirement to replace a field with a character as per the length of the field. Suppose i have a file where second field is of 20 character length. I want to replace second field with 20 stars (*). like ******************** As the field is not a fixed one, i want to do the... (2 Replies)
Discussion started by: gani_85
2 Replies

9. Shell Programming and Scripting

Fixed length to delimited file conversion

Hi All, I need to convert a fixed length file to a delimited file with , (comma). But not all columns, some of the columns in the fixed files are used as fillers and I do not need that in the output file. test_fixed_len.txt I 0515 MR 394 I 0618 MR & MRS 942 I 0618 MR & MRS... (7 Replies)
Discussion started by: member2014
7 Replies

10. Programming

C: inputting string of unknown length

I realize this general issue (inputting strings of variable length in C) has been addressed in myriad locations before, but I'm interested in knowing why my specific approach is not working. (BTW I'm intentionally keeping the size increments small so that I can more easily follow what's going on.... (5 Replies)
Discussion started by: DevuanFan
5 Replies
RS(1)							    BSD General Commands Manual 						     RS(1)

NAME
rs -- reshape a data array SYNOPSIS
rs [-[csCS][x] [kKgGw][N] tTeEnyjhHmz] [rows [cols]] DESCRIPTION
The rs utility reads the standard input, interpreting each line as a row of blank-separated entries in an array, transforms the array accord- ing to the options, and writes it on the standard output. With no arguments it transforms stream input into a columnar format convenient for terminal viewing. The shape of the input array is deduced from the number of lines and the number of columns on the first line. If that shape is inconvenient, a more useful one might be obtained by skipping some of the input with the -k option. Other options control interpretation of the input col- umns. The shape of the output array is influenced by the rows and cols specifications, which should be positive integers. If only one of them is a positive integer, rs computes a value for the other which will accommodate all of the data. When necessary, missing data are supplied in a manner specified by the options and surplus data are deleted. There are options to control presentation of the output columns, including transposition of the rows and columns. The following options are available: -cx Input columns are delimited by the single character x. A missing x is taken to be `^I'. -sx Like -c, but maximal strings of x are delimiters. -Cx Output columns are delimited by the single character x. A missing x is taken to be `^I'. -Sx Like -C, but padded strings of x are delimiters. -t Fill in the rows of the output array using the columns of the input array, that is, transpose the input while honoring any rows and cols specifications. -T Print the pure transpose of the input, ignoring any rows or cols specification. -kN Ignore the first N lines of input. -KN Like -k, but print the ignored lines. -gN The gutter width (inter-column space), normally 2, is taken to be N. -GN The gutter width has N percent of the maximum column width added to it. -e Consider each line of input as an array entry. -n On lines having fewer entries than the first line, use null entries to pad out the line. Normally, missing entries are taken from the next line of input. -y If there are too few entries to make up the output dimensions, pad the output by recycling the input from the beginning. Normally, the output is padded with blanks. -h Print the shape of the input array and do nothing else. The shape is just the number of lines and the number of entries on the first line. -H Like -h, but also print the length of each line. -j Right adjust entries within columns. -wN The width of the display, normally 80, is taken to be the positive integer N. -m Do not trim excess delimiters from the ends of the output array. -z Adapt column widths to fit the largest entries appearing in them. With no arguments, rs transposes its input, and assumes one array entry per input line unless the first non-ignored line is longer than the display width. Option letters which take numerical arguments interpret a missing number as zero unless otherwise indicated. EXAMPLES
The rs utility can be used as a filter to convert the stream output of certain programs (e.g., spell(1), du(1), file(1), look(1), nm(1), who(1), and wc(1)) into a convenient ``window'' format, as in % who | rs This function has been incorporated into the ls(1) program, though for most programs with similar output rs suffices. To convert stream input into vector output and back again, use % rs 1 0 | rs 0 1 A 10 by 10 array of random numbers from 1 to 100 and its transpose can be generated with % jot -r 100 | rs 10 10 | tee array | rs -T > tarray In the editor vi(1), a file consisting of a multi-line vector with 9 elements per line can undergo insertions and deletions, and then be neatly reshaped into 9 columns with :1,$!rs 0 9 Finally, to sort a database by the first line of each 4-line field, try % rs -eC 0 4 | sort | rs -c 0 1 SEE ALSO
jot(1), pr(1), sort(1), vi(1) BUGS
Handles only two dimensional arrays. The algorithm currently reads the whole file into memory, so files that do not fit in memory will not be reshaped. Fields cannot be defined yet on character positions. Re-ordering of columns is not yet possible. There are too many options. Multibyte characters are not recognized. BSD
July 30, 2004 BSD
All times are GMT -4. The time now is 04:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy