Sponsored Content
Top Forums Shell Programming and Scripting Extracting records with unique fields from a fixed width txt file Post 302165952 by KevinADC on Sunday 10th of February 2008 01:05:35 AM
Old 02-10-2008
You're welcome. Actually that data structure could have been a bit simpler:

Code:
while (<>){
   chomp;
   my ($first,$mi,$last,$state) = unpack("a10a2a11a2",$_);
   (s/^\s*//, s/\s*$//) for ($first,$mi,$last,$state);
   $names{"$first,$last"}{count}++;
   $names{"$first,$last"}{name} = "$first $mi $last $state",
}

This eliminates the need for the seperate hash to keep track of the counts. I like to use the seperate hash for counts because in general data is much more complex than this and incrementing a count can be much easier done if it is kept seperate.
 

10 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

2. Shell Programming and Scripting

summing up the fields in fixed width file

Hi, I have a fixed width file with some records as given below: " 1000Nalsdjflj243324jljlj" "-0300Njfowjljl309933fsf" " 0010Njsfsjklj342344fsl" I want to sum-up first field values(i.e from 2nd character to 6th character)of each record. so for the above file i want to add (1000 - 300+... (2 Replies)
Discussion started by: srilaxmi
2 Replies

3. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

4. Shell Programming and Scripting

Manupulating Records in a fixed width file

I am trying to determine what would be a fast and simple way to manipulate data that comes in a fixed width format. This data has 6 segments within a record. Each record needs to written out with a header and the 6 segments. Based on the value in column #6 the fields will be defined accordingly.... (4 Replies)
Discussion started by: Muga801
4 Replies

5. Shell Programming and Scripting

Comparing two fixed width file

Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228... (10 Replies)
Discussion started by: anshul_er
10 Replies

6. Shell Programming and Scripting

Apply condition on fixed width file and filter records

Dear members.. I have a fixed width file. Requirement is as below:- 1. Scan each record from this fixed width file 2. Check for value under field no "6" equals to "ABC". If yes, then filter this record into the output file Please suggest a unix command to achieve this, my guess awk might... (6 Replies)
Discussion started by: sureshg_sampat
6 Replies

7. Shell Programming and Scripting

variable fixed-width fields

Hi there, CTL Port IO Rate(IOPS) Read Rate(IOPS) Write Rate(IOPS) Read Hit(%) Write Hit(%) Trans. Rate(MB/S) Read Trans. Rate(MB/S) Write Trans. Rate(MB/S) 09:36:48 0 A 136 0 135 97 100 ... (6 Replies)
Discussion started by: gray380
6 Replies

8. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

9. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

10. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies
rand(3) 							      OpenSSL								   rand(3)

NAME
rand - pseudo-random number generator SYNOPSIS
#include <openssl/rand.h> int RAND_set_rand_engine(ENGINE *engine); int RAND_bytes(unsigned char *buf, int num); int RAND_pseudo_bytes(unsigned char *buf, int num); void RAND_seed(const void *buf, int num); void RAND_add(const void *buf, int num, int entropy); int RAND_status(void); int RAND_load_file(const char *file, long max_bytes); int RAND_write_file(const char *file); const char *RAND_file_name(char *file, size_t num); int RAND_egd(const char *path); void RAND_set_rand_method(const RAND_METHOD *meth); const RAND_METHOD *RAND_get_rand_method(void); RAND_METHOD *RAND_SSLeay(void); void RAND_cleanup(void); /* For Win32 only */ void RAND_screen(void); int RAND_event(UINT, WPARAM, LPARAM); DESCRIPTION
Since the introduction of the ENGINE API, the recommended way of controlling default implementations is by using the ENGINE API functions. The default RAND_METHOD, as set by RAND_set_rand_method() and returned by RAND_get_rand_method(), is only used if no ENGINE has been set as the default "rand" implementation. Hence, these two functions are no longer the recommened way to control defaults. If an alternative RAND_METHOD implementation is being used (either set directly or as provided by an ENGINE module), then it is entirely responsible for the generation and management of a cryptographically secure PRNG stream. The mechanisms described below relate solely to the software PRNG implementation built in to OpenSSL and used by default. These functions implement a cryptographically secure pseudo-random number generator (PRNG). It is used by other library functions for exam- ple to generate random keys, and applications can use it when they need randomness. A cryptographic PRNG must be seeded with unpredictable data such as mouse movements or keys pressed at random by the user. This is described in RAND_add(3). Its state can be saved in a seed file (see RAND_load_file(3)) to avoid having to go through the seeding process whenever the application is started. RAND_bytes(3) describes how to obtain random data from the PRNG. INTERNALS
The RAND_SSLeay() method implements a PRNG based on a cryptographic hash function. The following description of its design is based on the SSLeay documentation: First up I will state the things I believe I need for a good RNG. 1 A good hashing algorithm to mix things up and to convert the RNG 'state' to random numbers. 2 An initial source of random 'state'. 3 The state should be very large. If the RNG is being used to generate 4096 bit RSA keys, 2 2048 bit random strings are required (at a minimum). If your RNG state only has 128 bits, you are obviously limiting the search space to 128 bits, not 2048. I'm probably get- ting a little carried away on this last point but it does indicate that it may not be a bad idea to keep quite a lot of RNG state. It should be easier to break a cipher than guess the RNG seed data. 4 Any RNG seed data should influence all subsequent random numbers generated. This implies that any random seed data entered will have an influence on all subsequent random numbers generated. 5 When using data to seed the RNG state, the data used should not be extractable from the RNG state. I believe this should be a require- ment because one possible source of 'secret' semi random data would be a private key or a password. This data must not be disclosed by either subsequent random numbers or a 'core' dump left by a program crash. 6 Given the same initial 'state', 2 systems should deviate in their RNG state (and hence the random numbers generated) over time if at all possible. 7 Given the random number output stream, it should not be possible to determine the RNG state or the next random number. The algorithm is as follows. There is global state made up of a 1023 byte buffer (the 'state'), a working hash value ('md'), and a counter ('count'). Whenever seed data is added, it is inserted into the 'state' as follows. The input is chopped up into units of 20 bytes (or less for the last block). Each of these blocks is run through the hash function as fol- lows: The data passed to the hash function is the current 'md', the same number of bytes from the 'state' (the location determined by in incremented looping index) as the current 'block', the new key data 'block', and 'count' (which is incremented after each use). The result of this is kept in 'md' and also xored into the 'state' at the same locations that were used as input into the hash function. I believe this system addresses points 1 (hash function; currently SHA-1), 3 (the 'state'), 4 (via the 'md'), 5 (by the use of a hash function and xor). When bytes are extracted from the RNG, the following process is used. For each group of 10 bytes (or less), we do the following: Input into the hash function the local 'md' (which is initialized from the global 'md' before any bytes are generated), the bytes that are to be overwritten by the random bytes, and bytes from the 'state' (incrementing looping index). From this digest output (which is kept in 'md'), the top (up to) 10 bytes are returned to the caller and the bottom 10 bytes are xored into the 'state'. Finally, after we have finished 'num' random bytes for the caller, 'count' (which is incremented) and the local and global 'md' are fed into the hash function and the results are kept in the global 'md'. I believe the above addressed points 1 (use of SHA-1), 6 (by hashing into the 'state' the 'old' data from the caller that is about to be overwritten) and 7 (by not using the 10 bytes given to the caller to update the 'state', but they are used to update 'md'). So of the points raised, only 2 is not addressed (but see RAND_add(3)). SEE ALSO
BN_rand(3), RAND_add(3), RAND_load_file(3), RAND_egd(3), RAND_bytes(3), RAND_set_rand_method(3), RAND_cleanup(3) 0.9.7d 2003-11-20 rand(3)
All times are GMT -4. The time now is 12:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy