Sponsored Content
Full Discussion: Data transformation
Top Forums Shell Programming and Scripting Data transformation Post 302939347 by Don Cragun on Tuesday 24th of March 2015 10:28:07 PM
Old 03-24-2015
Note that on many systems you may not be able to do much with your output file. All of the text processing tools you're used to using on files are only guaranteed to work on text files. And, by definition, text files can't contain more than LINE_MAX bytes per line (including the terminating <newline> character). You can find the value of LINE_MAX on your system using the command:
Code:
getconf LINE_MAX

but 2048 is a common value for this limit.

UNIX systems are frequently more concerned about LINE_MAX than Linux systems, but even when the text processing utilities can handle "unlimited" line lengths, visualizing lines that long is hard for humans. What do you intend to do with the output produced by this script?

What operating system are you using?
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Transformation capital letter

:confused: Hye everybody i would like to know if exist a internet site where i can founs some interesting shell script very usefull I need to transform hundreds names of files escribed in CAPITAL letter in minuscule letter do oyu know a mean o do that that thanks to a script or a shell... (1 Reply)
Discussion started by: Dark Angel
1 Replies

2. Shell Programming and Scripting

file name transformation

I've got a multitude of text data files that carry exactly the same kind of data. Unfortunately some of them have a different filename format some are: 'category'_'month'-'year'_act.txt an example being: daf_Apr-1961_act.txt and some are: 'category'_ 'year'-'month'_act.txt an... (16 Replies)
Discussion started by: vrms
16 Replies

3. Shell Programming and Scripting

text transformation with sed or awk

Hi there, I'm trying to extract automatically opening hours from a website. The page displaying the schedules is http://www.natureetdecouvertes.com/pages/gener/view_FO_STORE_corgen.asp?mag_cod=xxx with xxx going from 101 to 174 I managed to get the following output : le lundi de 10.30 à... (4 Replies)
Discussion started by: chebarbudo
4 Replies

4. Shell Programming and Scripting

XML to csv transformation

Hi, I want to write a perl script. Which should accept the xml file, one xsl file and the loaction. The perl script should process the xml file using the xsl file and puts the out put in specified location. For example: My.perl is perls cript. my.xml is like this <?xml version="1.0"... (2 Replies)
Discussion started by: siba.s.nayak
2 Replies

5. Shell Programming and Scripting

xslt transformation through Unix

Hi .. I have one input XML and I want to convert into another XML using parameter mapping through Database through Unix shell script. But I dont have idea how to do that. And how can I create xsl sheet if mapping is through database tables. Please help me on this. (1 Reply)
Discussion started by: srinu19
1 Replies

6. UNIX for Advanced & Expert Users

Need help in xslt transformation

Hi I have one input xml file <param name="EXTR_COL" valueDesc="AUTHD_RFLL" value="rx.AUTHD_RFLL" /> There is a mapping parameters in Database. if EXTR_COL is present in input XML then it is mapped to fieldlist. so the o/p XML looks like <fieldlist> <datasource... (1 Reply)
Discussion started by: srinu19
1 Replies

7. Shell Programming and Scripting

XML file transformation

Hi all, I have to transform a XML file like this: <?xml version="1.0"?> <vocabulary> <voc_id>102</voc_id> <name>Vocabulary Name</name> <description>Voc description</description> <relations>3</relations> <hierarchy>5</hierarchy> <word> <word_id>1</word_id> ... (1 Reply)
Discussion started by: aLittleBeat
1 Replies

8. Shell Programming and Scripting

Clipboard transformation scripting

Hello all, I've done a bit of clipboard transformation scripting using xclip before, piping contents with " xclip -o -selection clipboard " to grep, sed, awk, then back into the clipboard with " xclip -i -selection clipboard " ... but I am not a fantastically skilled user of either of the three... (4 Replies)
Discussion started by: la2ar0
4 Replies

9. Shell Programming and Scripting

Row to Column transformation

Hello Experts, I need to transform rows into column using awk. I tried few things but failed to obtain desired output, as I'm fairly new to awk. i/p file 100, READ, 12 100, WRITE, 8 100, SEEK, 1 142, READ, 2 142, WRITE, 34 142, SEEK, 3 O/p Needed PROC_ID 100 142 READ 12 ... (2 Replies)
Discussion started by: sybadm
2 Replies
PASTE(1P)						     POSIX Programmer's Manual							 PASTE(1P)

PROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the correspond- ing Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux. NAME
paste - merge corresponding or subsequent lines of files SYNOPSIS
paste [-s][-d list] file... DESCRIPTION
The paste utility shall concatenate the corresponding lines of the given input files, and write the resulting lines to standard output. The default operation of paste shall concatenate the corresponding lines of the input files. The <newline> of every line except the line from the last input file shall be replaced with a <tab>. If an end-of-file condition is detected on one or more input files, but not all input files, paste shall behave as though empty lines were read from the files on which end-of-file was detected, unless the -s option is specified. OPTIONS
The paste utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines. The following options shall be supported: -d list Unless a backslash character appears in list, each character in list is an element specifying a delimiter character. If a backslash character appears in list, the backslash character and one or more characters following it are an element specifying a delimiter character as described below. These elements specify one or more delimiters to use, instead of the default <tab>, to replace the <newline> of the input lines. The elements in list shall be used circularly; that is, when the list is exhausted the first element from the list is reused. When the -s option is specified: * The last <newline> in a file shall not be modified. * The delimiter shall be reset to the first element of list after each file operand is processed. When the -s option is not specified: * The <newline>s in the file specified by the last file operand shall not be modified. * The delimiter shall be reset to the first element of list each time a line is processed from each file. If a backslash character appears in list, it and the character following it shall be used to represent the following delimiter characters: <newline>. <tab>. \ Backslash character. Empty string (not a null character). If '' is immediately followed by the character 'x', the character 'X', or any character defined by the LC_CTYPE digit keyword (see the Base Definitions volume of IEEE Std 1003.1-2001, Chapter 7, Locale), the results are unspecified. If any other characters follow the backslash, the results are unspecified. -s Concatenate all of the lines of each separate input file in command line order. The <newline> of every line except the last line in each input file shall be replaced with the <tab>, unless otherwise specified by the -d option. OPERANDS
The following operand shall be supported: file A pathname of an input file. If '-' is specified for one or more of the files, the standard input shall be used; the standard input shall be read one line at a time, circularly, for each instance of '-' . Implementations shall support pasting of at least 12 file operands. STDIN
The standard input shall be used only if one or more file operands is '-' . See the INPUT FILES section. INPUT FILES
The input files shall be text files, except that line lengths shall be unlimited. ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of paste: LANG Provide a default value for the internationalization variables that are unset or null. (See the Base Definitions volume of IEEE Std 1003.1-2001, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.) LC_ALL If set to a non-empty string value, override the values of all the other internationalization variables. LC_CTYPE Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files). LC_MESSAGES Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error. NLSPATH Determine the location of message catalogs for the processing of LC_MESSAGES . ASYNCHRONOUS EVENTS
Default. STDOUT
Concatenated lines of input files shall be separated by the <tab> (or other characters under the control of the -d option) and terminated by a <newline>. STDERR
The standard error shall be used only for diagnostic messages. OUTPUT FILES
None. EXTENDED DESCRIPTION
None. EXIT STATUS
The following exit values shall be returned: 0 Successful completion. >0 An error occurred. CONSEQUENCES OF ERRORS
If one or more input files cannot be opened when the -s option is not specified, a diagnostic message shall be written to standard error, but no output is written to standard output. If the -s option is specified, the paste utility shall provide the default behavior described in Utility Description Defaults . The following sections are informative. APPLICATION USAGE
When the escape sequences of the list option-argument are used in a shell script, they must be quoted; otherwise, the shell treats the '' as a special character. Conforming applications should only use the specific backslash escaped delimiters presented in this volume of IEEE Std 1003.1-2001. Histor- ical implementations treat 'x', where 'x' is not in this list, as 'x', but future implementations are free to expand this list to recog- nize other common escapes similar to those accepted by printf and other standard utilities. Most of the standard utilities work on text files. The cut utility can be used to turn files with arbitrary line lengths into a set of text files containing the same data. The paste utility can be used to create (or recreate) files with arbitrary line lengths. For example, if file contains long lines: cut -b 1-500 -n file > file1 cut -b 501- -n file > file2 creates file1 (a text file) with lines no longer than 500 bytes (plus the <newline>) and file2 that contains the remainder of the data from file. Note that file2 is not a text file if there are lines in file that are longer than 500 + {LINE_MAX} bytes. The original file can be recreated from file1 and file2 using the command: paste -d "" file1 file2 > file The commands: paste -d "" ... paste -d "" ... are not necessarily equivalent; the latter is not specified by this volume of IEEE Std 1003.1-2001 and may result in an error. The con- struct '' is used to mean "no separator" because historical versions of paste did not follow the syntax guidelines, and the command: paste -d"" ... could not be handled properly by getopt(). EXAMPLES
1. Write out a directory in four columns: ls | paste - - - - 2. Combine pairs of lines from a file into single lines: paste -s -d " " file RATIONALE
None. FUTURE DIRECTIONS
None. SEE ALSO
Utility Description Defaults, cut, grep, pr COPYRIGHT
Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol- ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html . IEEE
/The Open Group 2003 PASTE(1P)
All times are GMT -4. The time now is 07:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy