Sponsored Content
Top Forums Shell Programming and Scripting Convert fixed value fields to comma separated values Post 303020111 by mad man on Friday 13th of July 2018 01:33:24 PM
Old 07-13-2018
Convert fixed value fields to comma separated values

Hi All,

Hope you are doing Great!!!.

Today i have came up with a problem to say exactly it was for performance improvement.

I have written code in perl as a solution for this to cut in specific range, but it is taking time to run for files thousands of lines so i am expecting

a sed command kind of thing to make it run quickly.

input format:

Code:
--> TP  ID: TEST TP                        XLATE KEY:   ANSIXX99  AL:   D
      INT ID: TESTREFORMAT            XLATE TABLE: X820XR99
      DOC ID: 820    DIR: I STD: ANSI     COM: X      VERS: NONCTX       STAT: P

--> TP  ID: TEST TP                             XLATE KEY:   ANSIXX41  AL:   D
      INT ID: TESTREFORMAT                        XLATE TABLE: X820XR99
      DOC ID: 820    DIR: I STD: ANSI     COM: X      VERS: 004010       STAT: P

--> TP  ID: TEST TP                             XLATE KEY:   XXXXXXXX   AL:   D
     INT ID: TESTREFORMAT                        XLATE TABLE: XXXXXXXX
     DOC ID: 820    DIR: I STD: ANSI     COM: X      VERS: 004010       STAT: T

output format required:

Code:
TEST TP,ANSIXX99,D,TESTREFORMAT,X820XR99,820,I,ANSI,X,NONCTX,P
TEST TP,ANSIXX41,D,TESTREFORMAT,X820XR99,820,I,ANSI,X,004010,P
TEST TP,XXXXXXXX,D,TESTREFORMAT,XXXXXXXX,820,I,ANSI,X,004010,T

I have a file with input format given as example in above.

Rules for input format:
  • The "--> TP ID:" will repeat for every three lines.
    All the values after ":" were variable in length with in length given to it but all the values before ":" was fixed in length.

So our motivation was to make the values after ":" to be like in output format with comma separated value.

I am using AIX V6.0 OS. SED command as solution will be a preferable .

Thanks.

---------- Post updated at 11:03 PM ---------- Previous update was at 11:01 PM ----------

I have pasted actual data but i could see a empty line in between every 3 lines but in actual data there will not be any empty lines in input file.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove whitespaces between comma separated fields from file

Hello all, I am a unix dummy. I am trying to remove spaces between fields. I have the file in the following format 12332432, 2345 , asdfsdf ,100216 , 9999999 12332431, 2341 , asdfsd2 ,100213 , 9999999 &... (2 Replies)
Discussion started by: nitinbjoshi
2 Replies

2. Shell Programming and Scripting

Parse apart strings of comma separated data with varying number of fields

I have a situation where I am reading a text file line-by-line. Those lines of data contain comma separated fields of data. However, each line can vary in the number of fields it can contain. What I need to do is parse apart each line and write each field of data found (left to right) into a file.... (7 Replies)
Discussion started by: 2reperry
7 Replies

3. Shell Programming and Scripting

Extracting the values separated by comma

Hi, I have a variable which has a list of string separated by comma. for ex , Variable=/usr/bin,/usr/smrshbin,/tmp How can i get the values between the commas separately using shell scripts.Please help me. Thanks, Padmini. (6 Replies)
Discussion started by: padmisri
6 Replies

4. Shell Programming and Scripting

To agregate Comma separated values

Hi pls help me to get the code: i have a file in which content is : 2.01304E+11 2.01304E+11 ori 2 01:00 2.01304E+11 2.01304E+11 ori 2 01:02 2.01304E+11 2.01304E+11 ori 3 01:02 2.01304E+11 2.01304E+11 ori 3 ... (7 Replies)
Discussion started by: Aditya.Gurgaon
7 Replies

5. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

6. Shell Programming and Scripting

Convert comma separated file to fix length

Hi, I am converting a comma separated file to fixed field lenght and I am using that: COLUMNS="25 24 67 26 39 63 20 34 35 14 397" ( cat $indir/input_file.dat | \ $AWK -v columns="$COLUMNS" ' BEGIN { FS=","; OFS=""; split(columns, arr, " "); } { for(i=1; i<=NF;... (5 Replies)
Discussion started by: apenkov
5 Replies

7. Shell Programming and Scripting

Needs help in parsing comma separated values

hello experts, i am retrieving values in variables jobKey and jobName within my shell script. these values are returned to me within braces and i am using following command to remove those braces: jobKeys=`echo $jobKeys | sed 's:^.\(.*\).$:\1:'` jobNames=`echo $jobNames | sed... (1 Reply)
Discussion started by: avikaljain
1 Replies

8. Shell Programming and Scripting

Convert column to quote and comma separated row

Hi, I have a list of tables in a file.txt C_CLAIM C_HLD C_PROVIDER I want the output to be 'C_CLAIM','C_HLD','C_PROVIDER' Currently I'm usin awk and getting output which is almost correct but still has minor defects awk -vORS="','" '{ print $1 }' file.txt The output of... (4 Replies)
Discussion started by: wahi80
4 Replies

9. Shell Programming and Scripting

Comma separated values to individual lines

My OS : RHEL 6.7 I have a text file with comma separated values like below $ cat testString.txt 'JOHN' , 'KEITH' , 'NEWMAN' , 'URSULA' , 'ARIANNA' , 'CHENG', . . . . I want these values to appear like below 'JOHN' , 'KEITH' , 'NEWMAN' , 'URSULA' , 'ARIANNA' , 'CHENG', .... (4 Replies)
Discussion started by: kraljic
4 Replies

10. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
cut(1)							      General Commands Manual							    cut(1)

Name
       cut - cut out selected fields of each line of a file

Syntax
       cut -clist [file1 file2...]
       cut -flist [-dchar] [-s] [file1 file2...]

Description
       Use  the  command to cut out columns from a table or fields from each line of a file.  The fields as specified by list can be fixed length,
       that is, character positions as on a punched card (-c option), or the length can vary from line to line and be marked with a  field  delim-
       iter character like tab (-f option).  The command can be used as a filter.  If no files are given, the standard input is used.

       Use to make horizontal ``cuts'' (by context) through a file, or to put files together in columns.  To reorder columns in a table, use and

Options
       list	   Specifies  ranges  that must be a comma-separated list of integer field numbers in increasing order.  With optional - indicates
		   ranges as in the -o option of nroff/troff for page ranges; for example, 1,4,7; 1-3,8; -5,10 (short for 1-5,10);  or	3-  (short
		   for third through last field).

       -clist	   Specifies character positions to be cut out.  For example, -c1-72 would pass the first 72 characters of each line.

       -flist	   Specifies  the  fields  to be cut out.  For example, -f1,7 copies the first and seventh field only.	Lines with no field delim-
		   iters are passed through intact (useful for table subheadings), unless -s is specified.

       -dchar	   Uses the specified character as the field delimiter.  Default is tab.  Space or other characters with special  meaning  to  the
		   shell must be quoted.  The -d option is used only in combination with the -f option, according to XPG3 and SVID2/SVID3.

       -s	   Suppresses  lines  with  no	delimiter  characters.	 Unless  specified, lines with no delimiters are passed through untouched.
		   Either the -c or -f option must be specified.

Examples
       Mapping of user IDs to names:
       cut -d: -f1,5 /etc/passwd
       To set name to the current login name for the csh shell:
       set name=`who am i | cut -f1 -d" "`
       To set name to the current login name for the sh, sh5, and ksh shells:
       name=`who am i | cut -f1 -d" "`

Diagnostics
       "line too long"	   A line can have no more than 511 characters or fields.

       "bad list for c/f option"
			   Missing -c or -f option or incorrectly specified list.  No error occurs if a line has fewer fields than the list  calls
			   for.

       "no fields"	   The list is empty.

See Also
       grep(1), paste(1)

																	    cut(1)
All times are GMT -4. The time now is 10:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy