Sponsored Content
Top Forums Shell Programming and Scripting Divide an EBCDIC files into multiple files based on value at 45-46 bytes Post 302775339 by Chubler_XL on Monday 4th of March 2013 03:57:51 PM
Old 03-04-2013
Not sure how many record types you have, this solution works if it's only a few just add more >(awk ...) args to the tee command

Code:
dd if=infile conv=ascii 2> /dev/null | tee >(
awk 'substr($0,44,2)=="12"' | dd of=employee.out conv=ebcdic 2> /dev/null ) >(
awk 'substr($0,44,2)=="15"' | dd of=customer.out conv=ebcdic 2> /dev/null ) |
awk 'substr($0,44,2)=="14"' | dd of=salary.out   conv=ebcdic 2> /dev/null


Last edited by Chubler_XL; 03-04-2013 at 05:03 PM..
This User Gave Thanks to Chubler_XL For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing EBCDIC files

Hi Guys, I wish to compare two ebcdic files. diff utility manual says it only compares two text files line by line.. I doubt this will be good for ebcdic files. cmp utility does binary comparision but I do not find any thing in manual referring if it does support ebcdic file format. let... (2 Replies)
Discussion started by: RishiPahuja
2 Replies

2. Shell Programming and Scripting

how to divide single large log file into multiple files.

Can you please help me with writing script for following purpose. I have to divide single large web access log file into multiple log files based on dates inside the log file. For example: if data is logged in the access file for jan-10-08 , jan-11-08 , Jan-12-08 then make small log file... (1 Reply)
Discussion started by: kamleshm
1 Replies

3. Shell Programming and Scripting

awk 3 files to one based on multiple columns

Hi all, I have three files, one is a navigation file, one is a depth file and one is a file containing the measured field of gravity. The formats of the files are; navigation file: 2006 320 17 39 0 0 *nav 21.31542 -157.887 2006 320 17 39 10 0 *nav 21.31542 -157.887 2006 320 17 39 20 0... (2 Replies)
Discussion started by: andrealphus
2 Replies

4. UNIX for Dummies Questions & Answers

Joining files based on multiple keys

I need a script (perl or awk..anything is fine) to join 3 files based on three key columns. The no of non-key columns can vary in each file. The columns are delimited by semicolon. For example, File1 Dim1;Dim2;Dim3;Fact1;Fact2;Fact3;Fact4;Fact5 ---- data delimited by semicolon --- ... (1 Reply)
Discussion started by: Sebben
1 Replies

5. UNIX for Advanced & Expert Users

Create a file based on multiple files

Hey everyone. I am trying to figure out a way to create a file that will be renamed based off of one of multiple files. For example, if I have 3 files (cat.ctl, dog.ctl, and bird.ctl) that gets placed on to an ftp site I want to create a single file called new.cat.ctl, new.dog.ctl, etc for each... (3 Replies)
Discussion started by: coach5779
3 Replies

6. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

7. Shell Programming and Scripting

want to concatenate multiple files based on the rest of ls -lrt

uadm@4132> ls -lrt -rw------- 1 uadm uadm 3811819 Jun 6 04:08 data_log-2010.05.30-10:04:08.txt -rw------- 1 uadm uadm 716246 Jun 13 01:38 data_log-2010.06.06-10:04:08.txt -rw------- 1 uadm uadm 996 Jun 13 04:00 data_log-2010.06.06-10:04:22.txt -rw------- 1 uadm uadm 7471 Jun 20 02:03... (5 Replies)
Discussion started by: mail2sant
5 Replies

8. Shell Programming and Scripting

Help with Archiving multiple files based on name and date

Dear Gurus, I am a novice in shell scripts. I have a requirement where I need to move files every day from Current Folder to Archive folder. Daily I will be receiving 5 files in the folder - /opt/data/feeds/. The feeds folder has two sub-folders - Current and Archive. For example the... (25 Replies)
Discussion started by: shankar1dada
25 Replies

9. UNIX for Dummies Questions & Answers

divide the file into multiple files based on the city name

Hi, I have a file abc.dat. It contains the fileds of empid, empname, empcity. each city contains 10 records. i want to create the city file and pass the same city records into the file. I don't know the city names. In unix using awk command how can we do? abc.dat: 1 john delhi 2... (2 Replies)
Discussion started by: raghukreddy.ab
2 Replies

10. UNIX for Advanced & Expert Users

Moving multiple files based on the pattern

I want to search for a particular file name patterns and move them to a specific folder, is it possible to do it with awk or sed? (1 Reply)
Discussion started by: rudoraj
1 Replies
dd(1M)							  System Administration Commands						    dd(1M)

NAME
dd - convert and copy a file SYNOPSIS
/usr/bin/dd [operand=value]... DESCRIPTION
The dd utility copies the specified input file to the specified output with possible conversions. The standard input and output are used by default. The input and output block sizes may be specified to take advantage of raw physical I/O. Sizes are specified in bytes; a number may end with k, b, or w to specify multiplication by 1024, 512, or 2, respectively. Numbers may also be separated by x to indicate multi- plication. The dd utility reads the input one block at a time, using the specified input block size. dd then processes the block of data actually returned, which could be smaller than the requested block size. dd applies any conversions that have been specified and writes the result- ing data to the output in blocks of the specified output block size. cbs is used only if ascii, asciib, unblock, ebcdic, ebcdicb, ibm, ibmb, or block conversion is specified. In the first two cases, cbs char- acters are copied into the conversion buffer, any specified character mapping is done, trailing blanks are trimmed, and a NEWLINE is added before sending the line to output. In the last three cases, characters up to NEWLINE are read into the conversion buffer and blanks are added to make up an output record of size cbs. ASCII files are presumed to contain NEWLINE characters. If cbs is unspecified or 0, the ascii, asciib, ebcdic, ebcdicb, ibm, and ibmb options convert the character set without changing the input file's block structure. The unblock and block options become a simple file copy. After completion, dd reports the number of whole and partial input and output blocks. OPERANDS
The following operands are supported: if=file Specifies the input path. Standard input is the default. of=file Specifies the output path. Standard output is the default. If the seek=expr conversion is not also specified, the output file will be truncated before the copy begins, unless conv=notrunc is specified. If seek=expr is specified, but conv=notrunc is not, the effect of the copy will be to preserve the blocks in the output file over which dd seeks, but no other portion of the output file will be pre- served. (If the size of the seek plus the size of the input file is less than the previous size of the output file, the output file is shortened by the copy.) ibs=n Specifies the input block size in n bytes (default is 512). obs=n Specifies the output block size in n bytes (default is 512). bs=n Sets both input and output block sizes to n bytes, superseding ibs= and obs=. If no conversion other than sync, noerror, and notrunc is specified, each input block is copied to the output as a single block without aggregating short blocks. cbs=n Specifies the conversion block size for block and unblock in bytes by n (default is 0). If cbs= is omitted or given a value of 0, using block or unblock produces unspecified results. This option is used only if ASCII or EBCDIC conversion is specified. For the ascii and asciib operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted. For the ebcdic, ebcdicb, ibm, and ibmb operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added. files=n Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device). skip=n Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded. iseek=n Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow). oseek=n Seeks n blocks from beginning of output file before copying. seek=n Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files. count=n Copies only n input blocks. conv=value[,value...] Where values are comma-separated symbols from the following list: ascii Converts EBCDIC to ASCII. asciib Converts EBCDIC to ASCII using BSD-compatible character translations. ebcdic Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ebcdicb Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEW- LINEs, sets up a pipeline with dd conv=unblock beforehand. ibm Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. ibmb Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand. The ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb) values are mutually exclusive. block Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEW- LINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported. unblock Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character. The block and unblock values are mutually exclusive. lcase Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion. ucase Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion. The lcase and ucase symbols are mutually exclusive. swab Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored. noerror Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If the sync conversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output. notrunc Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the preceding of=file operand.) sync Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.) If operands other than conv= are specified more than once, the last specified operand=value is used. For the bs=, cbs=, ibs=, and obs= operands, the application must supply an expression specifying a size in bytes. The expression, expr, can be: 1. a positive decimal number 2. a positive decimal number followed by k, specifying multiplication by 1024 3. a positive decimal number followed by b, specifying multiplication by 512 4. two or more positive decimal numbers (with or without k or b) separated by x, specifying the product of the indicated values. All of the operands will be processed before any input is read. USAGE
See largefile(5) for the description of the behavior of dd when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). EXAMPLES
Example 1 Copying from one tape drive to another The following example copies from tape drive 0 to tape drive 1, using a common historical device naming convention. example% dd if=/dev/rmt/0h of=/dev/rmt/1h Example 2 Stripping the first 10 bytes from standard input The following example strips the first 10 bytes from standard input: example% dd ibs=10 skip=1 Example 3 Reading a tape into an ASCII file This example reads an EBCDIC tape blocked ten 80-byte EBCDIC card images per block into the ASCII file x: example% dd if=/dev/tape of=x ibs=800 cbs=80 conv=ascii,lcase Example 4 Using conv=sync to write to tape The following example uses conv=sync when writing to a tape: example% tar cvf - . | compress | dd obs=1024k of=/dev/rmt/0 conv=sync ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of dd: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 The input file was copied successfully. >0 An error occurred. If an input error is detected and the noerror conversion has not been specified, any partial output block will be written to the output file, a diagnostic message will be written, and the copy operation will be discontinued. If some other error is detected, a diagnostic mes- sage will be written and the copy operation will be discontinued. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cp(1), sed(1), tr(1), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
f+p records in(out) numbers of full and partial blocks read(written) NOTES
Do not use dd to copy files between file systems having different block sizes. Using a blocked device to copy a file will result in extra nulls being added to the file to pad the final block to the block boundary. When dd reads from a pipe, using the ibs=X and obs=Y operands, the output will always be blocked in chunks of size Y. When bs=Z is used, the output blocks will be whatever was available to be read from the pipe at the time. When using dd to copy files to a tape device, the file size must be a multiple of the device sector size (for example, 512 Kbyte). To copy files of arbitrary size to a tape device, use tar(1) or cpio(1). For SIGINT, dd writes status information to standard error before exiting. It takes the standard action for all other signals. SunOS 5.11 16 Sep 1996 dd(1M)
All times are GMT -4. The time now is 09:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy