Sponsored Content
Top Forums Shell Programming and Scripting add character to every end-of line in file Post 302365185 by methyl on Monday 26th of October 2009 09:01:01 AM
Old 10-26-2009
Assuming this is a file containing no record delimiters whatsoever and that all output records will be exactly 47 bytes.

I believe that the record will be too long for "awk" (not checked). We can use the "dd" command to break the long string into 47-byte chunks.
Input block size (ibs=) chosen is deliberately larger than 700 "records" times 47 bytes (32,900 bytes).


Code:
dd if=inputfile.txt of=outputfile.txt ibs=131072 cbs=47 conv=unblock

At this point outputfile.txt has 47-byte records each terminated with the unix standard text file line terminator (line-feed).

In the unlikely event that you really do need to convert the line-feed characters (octal 012) to carriage-return characters (octal 015).

Code:
cat outputfile.txt | tr '\n' '\r' > outputfile2.txt

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Deleting end of line $ character in file

Hi, I've got a file where in the middle of the record is a $ end of line character, visible only when I open the file in vi and do :set list. How to I get rid of the character in the middle and keep it at the end. The middle $ character always appears after SW, so that can be used to tag it.... (3 Replies)
Discussion started by: bwrynz1
3 Replies

2. UNIX for Dummies Questions & Answers

removing a character and addending to end in each line in a file

HI i am having a file this (sys19:pnlfct:/pfact/temp>) cat temp_sand 1234567890 1234567890 1234567890 1234567890 I want to make this file as (sys19:pnlfct:/pfact/temp>) cat temp_sand 1456789023 1456789023 1456789023 1456789023 just take the 2nd and 3rd position and put it... (5 Replies)
Discussion started by: arunkumar_mca
5 Replies

3. UNIX for Dummies Questions & Answers

How to add new line character at the end of a file

hi all, i have this question: How to add new line character at the end of a file???? i need this because i am loading a file to sybase and i have problems with the last record thanks for your help (5 Replies)
Discussion started by: DebianJ
5 Replies

4. Shell Programming and Scripting

append a character at end of each line of a file

Hi, i want to append a character '|' at end of each line of a file abc.txt. for example if the file abc.txt conatins: a|b|c 1|2|33 w|2|11 i want result file xyz.txt a|b|c| 1|2|33| w|2|11| I know this is simple but sumhow i am not able to reach end of line. its urgent, thanks for... (4 Replies)
Discussion started by: muaz
4 Replies

5. Shell Programming and Scripting

add character to the end of each line in file

hi all i have 32 lines in file. the length of each line is 82 , i want that in the end of each line , means in postion 83-84 to put two characters 0d(=\015), 0a(=\012) i want that the 0d will be in postion 83 and the 0a will be in postion 84 in each line of the file how shall i do it ? ... (7 Replies)
Discussion started by: naamas03
7 Replies

6. Shell Programming and Scripting

How to remove new line character at end of file.

I need to remove new line character from end of file. Suppose here are content. a|b|c|d|r a|b|c|d|r a|b|c|d|r <new line> that means file contains 4 lines but data is there in 3 lines. so I want that only 3 lines should be there in file. Please help (20 Replies)
Discussion started by: varun940
20 Replies

7. Shell Programming and Scripting

How to Remove comma as last character in end of last line of file?

how to Remove comma as last charector in end of last line of file: example: input file --------------- aaaaaa, bbbbbb, cccc, 12345, ____________ output file : ----------- aaaaaa, bbbbbb, (6 Replies)
Discussion started by: RahulJoshi
6 Replies

8. Shell Programming and Scripting

How to add a character at end of line?

Hai, I have got a small requirement in my script. and i am using bash shell. I need to add a dot (.) for some particular line in a file. Say for example, $Cat rmfile 1 This is line1 2 This is line2 3 This is line3 O/p should be : $Cat rmfile 1 This is line1 2 This is line2. #... (2 Replies)
Discussion started by: Sivajee
2 Replies

9. Shell Programming and Scripting

How to add newline character at end of file?

Hi All, I have following piece of code in UNIX C Shell script and I want to add one more command which can add newline at the end of file only if there is no newline character exists. foreach file (`ls $dd_PLAYCARD_EDI_IN`) if ( -f $dd_PLAYCARD_EDI_IN/${file} ) then cat -n... (4 Replies)
Discussion started by: jnrohit2k
4 Replies

10. Shell Programming and Scripting

Conditionally add character at end of line

Hi, I would like have a shell script to check every line in a file to see if it ends with ";". If this is NOT the last character ";" should be added. MyFile.csv : web9331801;01/01/2014 23:39:35;;"93962";353150256; web9331802;01/01/2014 23:44:29;;"479288";353153538; web9331803;01/01/2014... (14 Replies)
Discussion started by: vg77
14 Replies
dd(1M)							  System Administration Commands						    dd(1M)

NAME
dd - convert and copy a file SYNOPSIS
/usr/bin/dd [operand=value]... DESCRIPTION
The dd utility copies the specified input file to the specified output with possible conversions. The standard input and output are used by default. The input and output block sizes may be specified to take advantage of raw physical I/O. Sizes are specified in bytes; a number may end with k, b, or w to specify multiplication by 1024, 512, or 2, respectively. Numbers may also be separated by x to indicate multi- plication. The dd utility reads the input one block at a time, using the specified input block size. dd then processes the block of data actually returned, which could be smaller than the requested block size. dd applies any conversions that have been specified and writes the result- ing data to the output in blocks of the specified output block size. cbs is used only if ascii, asciib, unblock, ebcdic, ebcdicb, ibm, ibmb, or block conversion is specified. In the first two cases, cbs char- acters are copied into the conversion buffer, any specified character mapping is done, trailing blanks are trimmed, and a NEWLINE is added before sending the line to output. In the last three cases, characters up to NEWLINE are read into the conversion buffer and blanks are added to make up an output record of size cbs. ASCII files are presumed to contain NEWLINE characters. If cbs is unspecified or 0, the ascii, asciib, ebcdic, ebcdicb, ibm, and ibmb options convert the character set without changing the input file's block structure. The unblock and block options become a simple file copy. After completion, dd reports the number of whole and partial input and output blocks. OPERANDS
The following operands are supported: if=file Specifies the input path. Standard input is the default. of=file Specifies the output path. Standard output is the default. If the seek=expr conversion is not also specified, the output file will be truncated before the copy begins, unless conv=notrunc is specified. If seek=expr is specified, but conv=notrunc is not, the effect of the copy will be to preserve the blocks in the output file over which dd seeks, but no other portion of the output file will be pre- served. (If the size of the seek plus the size of the input file is less than the previous size of the output file, the output file is shortened by the copy.) ibs=n Specifies the input block size in n bytes (default is 512). obs=n Specifies the output block size in n bytes (default is 512). bs=n Sets both input and output block sizes to n bytes, superseding ibs= and obs=. If no conversion other than sync, noerror, and notrunc is specified, each input block is copied to the output as a single block without aggregating short blocks. cbs=n Specifies the conversion block size for block and unblock in bytes by n (default is 0). If cbs= is omitted or given a value of 0, using block or unblock produces unspecified results. This option is used only if ASCII or EBCDIC conversion is specified. For the ascii and asciib operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted. For the ebcdic, ebcdicb, ibm, and ibmb operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added. files=n Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device). skip=n Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded. iseek=n Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow). oseek=n Seeks n blocks from beginning of output file before copying. seek=n Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files. count=n Copies only n input blocks. conv=value[,value...] Where values are comma-separated symbols from the following list: ascii Converts EBCDIC to ASCII. asciib Converts EBCDIC to ASCII using BSD-compatible character translations. ebcdic Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ebcdicb Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEW- LINEs, sets up a pipeline with dd conv=unblock beforehand. ibm Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ibmb Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. The ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb) values are mutually exclusive. block Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEW- LINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported. unblock Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character. The block and unblock values are mutually exclusive. lcase Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion. ucase Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion. The lcase and ucase symbols are mutually exclusive. swab Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored. noerror Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If the sync conversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output. notrunc Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the preceding of=file operand.) sync Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.) If operands other than conv= are specified more than once, the last specified operand=value is used. For the bs=, cbs=, ibs=, and obs= operands, the application must supply an expression specifying a size in bytes. The expression, expr, can be: 1. a positive decimal number 2. a positive decimal number followed by k, specifying multiplication by 1024 3. a positive decimal number followed by b, specifying multiplication by 512 4. two or more positive decimal numbers (with or without k or b) separated by x, specifying the product of the indicated values. All of the operands will be processed before any input is read. USAGE
See largefile(5) for the description of the behavior of dd when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). EXAMPLES
Example 1 Copying from one tape drive to another The following example copies from tape drive 0 to tape drive 1, using a common historical device naming convention. example% dd if=/dev/rmt/0h of=/dev/rmt/1h Example 2 Stripping the first 10 bytes from standard input The following example strips the first 10 bytes from standard input: example% dd ibs=10 skip=1 Example 3 Reading a tape into an ASCII file This example reads an EBCDIC tape blocked ten 80-byte EBCDIC card images per block into the ASCII file x: example% dd if=/dev/tape of=x ibs=800 cbs=80 conv=ascii,lcase Example 4 Using conv=sync to write to tape The following example uses conv=sync when writing to a tape: example% tar cvf - . | compress | dd obs=1024k of=/dev/rmt/0 conv=sync ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of dd: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 The input file was copied successfully. >0 An error occurred. If an input error is detected and the noerror conversion has not been specified, any partial output block will be written to the output file, a diagnostic message will be written, and the copy operation will be discontinued. If some other error is detected, a diagnostic mes- sage will be written and the copy operation will be discontinued. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cp(1), sed(1), tr(1), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
f+p records in(out) numbers of full and partial blocks read(written) NOTES
Do not use dd to copy files between file systems having different block sizes. Using a blocked device to copy a file will result in extra nulls being added to the file to pad the final block to the block boundary. When dd reads from a pipe, using the ibs=X and obs=Y operands, the output will always be blocked in chunks of size Y. When bs=Z is used, the output blocks will be whatever was available to be read from the pipe at the time. When using dd to copy files to a tape device, the file size must be a multiple of the device sector size (for example, 512 Kbyte). To copy files of arbitrary size to a tape device, use tar(1) or cpio(1). For SIGINT, dd writes status information to standard error before exiting. It takes the standard action for all other signals. SunOS 5.11 16 Sep 1996 dd(1M)
All times are GMT -4. The time now is 08:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy