Sponsored Content
Top Forums Shell Programming and Scripting Remove header(first line) and trailer(last line) in ANY given file Post 101838 by madhunk on Monday 13th of March 2006 02:36:40 PM
Old 03-13-2006
Thank you for the message. I was just wondering if we can do the same thing on EBCDIC data file which is fixed width. Can we remove the lines depending on the number of bytes in an EBCDIC file? Like remove 23 bytes in the beginning and 8 bytes at the end...

Thanks again,
Madhu
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

2. Shell Programming and Scripting

Removing Header & Trailer from a file

Hi All, I am karthik. I am new to this forum. I have one requirement. I have a file with header and footer. Header may be like HDR0001 or FILE20090110 (Assume it is unknown so far, but i am sure there is a header in the file) likewise file has the trailer too. I just... (7 Replies)
Discussion started by: karthi_gana
7 Replies

3. Shell Programming and Scripting

improve performance - replace $\| with $#@ and remove header and trailer records

Hi All, In my file i need to remove header and trailer records which comes in 1st line and last line respectively. After that i need to replace '$\|' with '$#@'. I am using sed command for this and its taking lot of time. Is there any other command which can be used to improve performance? ... (1 Reply)
Discussion started by: HemaV
1 Replies

4. Shell Programming and Scripting

Adding Header and Trailer records to a appended file

How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited I need append 3 files as above: File1: P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01 File 2:... (2 Replies)
Discussion started by: e1994264
2 Replies

5. UNIX for Dummies Questions & Answers

Adding header and trailer into a file

Hi, I want to add the below Header to all the files in sequence File1,File2,File3...etc "ABC,<number of chracter in the file>" e,g - If File1 is as below pqrstuvdt abcdefgh then I want to add the above header into it ,So that File1 becomes as below ABC,17 pqrstuvdt abcdefgh ... (9 Replies)
Discussion started by: spari2
9 Replies

6. Shell Programming and Scripting

Remove last few characters in a file but keeping Header and trailer intact

Hi All, I am trying write a simple command using AWK and SED to this but without any success. Here is what I am using: head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt Input: Header 1234567 abcdefgh... (2 Replies)
Discussion started by: nvuradi
2 Replies

7. Shell Programming and Scripting

Script to validate file header and trailer

Hi, I need a script that validates a file header/detail/trailer. File layout is: Header - Rec_Type|File_name|File_Date Detail - Rec_Type|field1|field2|field3... Trailder - Rec_Type|File_name|File_Date|Record_count Sample Data: HDR|customer_data.dat|20120709... (7 Replies)
Discussion started by: ash_sh
7 Replies

8. Shell Programming and Scripting

Verify the header and trailer in file

please see my requirement, I hope I am clear. (9 Replies)
Discussion started by: mirwasim
9 Replies

9. Shell Programming and Scripting

Split and add header and trailer from input file

I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file Also is it possible to output the TOM and PAT records in the same output file ? I need the output file names same as xyz_pattern_Datetimestamp.txt ... (23 Replies)
Discussion started by: techedipro
23 Replies
dd(1M)							  System Administration Commands						    dd(1M)

NAME
dd - convert and copy a file SYNOPSIS
/usr/bin/dd [operand=value]... DESCRIPTION
The dd utility copies the specified input file to the specified output with possible conversions. The standard input and output are used by default. The input and output block sizes may be specified to take advantage of raw physical I/O. Sizes are specified in bytes; a number may end with k, b, or w to specify multiplication by 1024, 512, or 2, respectively. Numbers may also be separated by x to indicate multi- plication. The dd utility reads the input one block at a time, using the specified input block size. dd then processes the block of data actually returned, which could be smaller than the requested block size. dd applies any conversions that have been specified and writes the result- ing data to the output in blocks of the specified output block size. cbs is used only if ascii, asciib, unblock, ebcdic, ebcdicb, ibm, ibmb, or block conversion is specified. In the first two cases, cbs char- acters are copied into the conversion buffer, any specified character mapping is done, trailing blanks are trimmed, and a NEWLINE is added before sending the line to output. In the last three cases, characters up to NEWLINE are read into the conversion buffer and blanks are added to make up an output record of size cbs. ASCII files are presumed to contain NEWLINE characters. If cbs is unspecified or 0, the ascii, asciib, ebcdic, ebcdicb, ibm, and ibmb options convert the character set without changing the input file's block structure. The unblock and block options become a simple file copy. After completion, dd reports the number of whole and partial input and output blocks. OPERANDS
The following operands are supported: if=file Specifies the input path. Standard input is the default. of=file Specifies the output path. Standard output is the default. If the seek=expr conversion is not also specified, the output file will be truncated before the copy begins, unless conv=notrunc is specified. If seek=expr is specified, but conv=notrunc is not, the effect of the copy will be to preserve the blocks in the output file over which dd seeks, but no other portion of the output file will be pre- served. (If the size of the seek plus the size of the input file is less than the previous size of the output file, the output file is shortened by the copy.) ibs=n Specifies the input block size in n bytes (default is 512). obs=n Specifies the output block size in n bytes (default is 512). bs=n Sets both input and output block sizes to n bytes, superseding ibs= and obs=. If no conversion other than sync, noerror, and notrunc is specified, each input block is copied to the output as a single block without aggregating short blocks. cbs=n Specifies the conversion block size for block and unblock in bytes by n (default is 0). If cbs= is omitted or given a value of 0, using block or unblock produces unspecified results. This option is used only if ASCII or EBCDIC conversion is specified. For the ascii and asciib operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted. For the ebcdic, ebcdicb, ibm, and ibmb operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added. files=n Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device). skip=n Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded. iseek=n Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow). oseek=n Seeks n blocks from beginning of output file before copying. seek=n Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files. count=n Copies only n input blocks. conv=value[,value...] Where values are comma-separated symbols from the following list: ascii Converts EBCDIC to ASCII. asciib Converts EBCDIC to ASCII using BSD-compatible character translations. ebcdic Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ebcdicb Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEW- LINEs, sets up a pipeline with dd conv=unblock beforehand. ibm Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ibmb Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. The ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb) values are mutually exclusive. block Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEW- LINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported. unblock Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character. The block and unblock values are mutually exclusive. lcase Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion. ucase Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion. The lcase and ucase symbols are mutually exclusive. swab Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored. noerror Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If the sync conversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output. notrunc Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the preceding of=file operand.) sync Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.) If operands other than conv= are specified more than once, the last specified operand=value is used. For the bs=, cbs=, ibs=, and obs= operands, the application must supply an expression specifying a size in bytes. The expression, expr, can be: 1. a positive decimal number 2. a positive decimal number followed by k, specifying multiplication by 1024 3. a positive decimal number followed by b, specifying multiplication by 512 4. two or more positive decimal numbers (with or without k or b) separated by x, specifying the product of the indicated values. All of the operands will be processed before any input is read. USAGE
See largefile(5) for the description of the behavior of dd when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). EXAMPLES
Example 1 Copying from one tape drive to another The following example copies from tape drive 0 to tape drive 1, using a common historical device naming convention. example% dd if=/dev/rmt/0h of=/dev/rmt/1h Example 2 Stripping the first 10 bytes from standard input The following example strips the first 10 bytes from standard input: example% dd ibs=10 skip=1 Example 3 Reading a tape into an ASCII file This example reads an EBCDIC tape blocked ten 80-byte EBCDIC card images per block into the ASCII file x: example% dd if=/dev/tape of=x ibs=800 cbs=80 conv=ascii,lcase Example 4 Using conv=sync to write to tape The following example uses conv=sync when writing to a tape: example% tar cvf - . | compress | dd obs=1024k of=/dev/rmt/0 conv=sync ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of dd: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 The input file was copied successfully. >0 An error occurred. If an input error is detected and the noerror conversion has not been specified, any partial output block will be written to the output file, a diagnostic message will be written, and the copy operation will be discontinued. If some other error is detected, a diagnostic mes- sage will be written and the copy operation will be discontinued. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cp(1), sed(1), tr(1), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
f+p records in(out) numbers of full and partial blocks read(written) NOTES
Do not use dd to copy files between file systems having different block sizes. Using a blocked device to copy a file will result in extra nulls being added to the file to pad the final block to the block boundary. When dd reads from a pipe, using the ibs=X and obs=Y operands, the output will always be blocked in chunks of size Y. When bs=Z is used, the output blocks will be whatever was available to be read from the pipe at the time. When using dd to copy files to a tape device, the file size must be a multiple of the device sector size (for example, 512 Kbyte). To copy files of arbitrary size to a tape device, use tar(1) or cpio(1). For SIGINT, dd writes status information to standard error before exiting. It takes the standard action for all other signals. SunOS 5.11 16 Sep 1996 dd(1M)
All times are GMT -4. The time now is 04:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy