Sponsored Content
Top Forums Shell Programming and Scripting Parsing large files in Solaris 11 Post 302952419 by Don Cragun on Tuesday 18th of August 2015 08:01:49 PM
Old 08-18-2015
Having an entry that is 78 bits long that contains characters is very strange. Most entries in a file are a stream of 8 bit bytes. So, to split your entries (each of which is 9.75 bytes) into 11 byte lines (your 9.75 bytes per entry plus 2 bits for byte packing and a newline so the output is a text file), you're probably going to find writing a C program to read bytes and rotate bits into the proper positions easier than doing it in a shell script.

What two bits should be added to your entries to produce 10 characters (assuming ASCII or EBCDIC) from your input entries?

If your entries are all 78 bits long, why is your grep looking for a varying number of characters before and after the colon and why is the string it is matching varying from 1 to 76 characters (not bits or bytes) inclusive instead of the 78 bits you specified???

Please show us the first 200 bytes of your input file piped through the command:
Code:
od -bcx

 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem with parsing a large file

Hi All, Following is the sample file and following is the op desired that is the last entry of each unique first field is required. My solution is as follows However the original file has around a million entries and around a 100,000 uniques first fields, so this soln.... (6 Replies)
Discussion started by: gauravgoel
6 Replies

2. Shell Programming and Scripting

Parsing a large log

I need to parse a large log say 300-400 mb The commands like awk and cat etc are taking time. Please help how to process. I need to process the log for certain values of current date. But I am unbale to do so. (17 Replies)
Discussion started by: asth
17 Replies

3. Shell Programming and Scripting

parsing large CDR XML file

Dear Freind in the file attached how parse the data to be like a normal table :D (3 Replies)
Discussion started by: saifsafaa
3 Replies

4. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

5. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

6. UNIX for Advanced & Expert Users

Need help with configuring large packet size on Solaris 7 / e6500

We're running Solaris 7 on FDDI n/w on an E6500 host and wish to use MTU (packet size) > 1500, more like 3072 bytes to begin with and possibly up to 4096 bytes. Linux has /etc/network/interfaces. Does ANYONE remember the equivalent in Unix? When I do ifconfig eth0 mtu 4000, I get the error... (0 Replies)
Discussion started by: sharique
0 Replies

7. Shell Programming and Scripting

Help needed for parsing large XML with awk.

My XML structure looks like: <?xml version="1.0" encoding="UTF-8"?> <SearchRepository> <SearchItems> <SearchItem> ... </SearchItem> <SearchItem> ... ... (1 Reply)
Discussion started by: jasonjustice
1 Replies

8. UNIX for Dummies Questions & Answers

How to display large file in Solaris?

Hi i want to see file in solaris which are eating space. like we have a listfiles command in AIX which show all the files in decreading order of the size . example of listfile command in this command i am able to all the huge file in root directory. do we have any similar command in... (1 Reply)
Discussion started by: scriptor
1 Replies

9. Shell Programming and Scripting

Parsing a subset of data from a large matrix

I do have a large matrix of the following format and it is tab delimited ch-ab1-20 ch-bb2-23 ch-ab1-34 ch-ab1-24 er-cc1-45 bv-cc1-78 ch-ab1-20 0 2 3 4 5 6 ch-bb2-23 3 0 5 ... (6 Replies)
Discussion started by: Kanja
6 Replies
encoding(n)						       Tcl Built-In Commands						       encoding(n)

__________________________________________________________________________________________________________________________________________________

NAME
encoding - Manipulate encodings SYNOPSIS
encoding option ?arg arg ...? _________________________________________________________________ INTRODUCTION
Strings in Tcl are encoded using 16-bit Unicode characters. Different operating system interfaces or applications may generate strings in other encodings such as Shift-JIS. The encoding command helps to bridge the gap between Unicode and these other formats. DESCRIPTION
Performs one of several encoding related operations, depending on option. The legal options are: encoding convertfrom ?encoding? data Convert data to Unicode from the specified encoding. The characters in data are treated as binary data where the lower 8-bits of each character is taken as a single byte. The resulting sequence of bytes is treated as a string in the specified encoding. If encoding is not specified, the current system encoding is used. encoding convertto ?encoding? string Convert string from Unicode to the specified encoding. The result is a sequence of bytes that represents the converted string. Each byte is stored in the lower 8-bits of a Unicode character. If encoding is not specified, the current system encoding is used. encoding names Returns a list containing the names of all of the encodings that are currently available. encoding system ?encoding? Set the system encoding to encoding. If encoding is omitted then the command returns the current system encoding. The system encod- ing is used whenever Tcl passes strings to system calls. EXAMPLE
It is common practice to write script files using a text editor that produces output in the euc-jp encoding, which represents the ASCII characters as singe bytes and Japanese characters as two bytes. This makes it easy to embed literal strings that correspond to non-ASCII characters by simply typing the strings in place in the script. However, because the source command always reads files using the ISO8859-1 encoding, Tcl will treat each byte in the file as a separate character that maps to the 00 page in Unicode. The resulting Tcl strings will not contain the expected Japanese characters. Instead, they will contain a sequence of Latin-1 characters that correspond to the bytes of the original string. The encoding command can be used to convert this string to the expected Japanese Unicode characters. For example, set s [encoding convertfrom euc-jp "xA4xCF"] would return the Unicode string "u306F", which is the Hiragana letter HA. SEE ALSO
Tcl_GetEncoding(3) KEYWORDS
encoding Tcl 8.1 encoding(n)
All times are GMT -4. The time now is 07:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy