Sponsored Content
Top Forums UNIX for Dummies Questions & Answers split a file with unique sets Post 302250116 by jim mcnamara on Wednesday 22nd of October 2008 05:56:22 PM
Old 10-22-2008
That is not what you asked for.... ):
- anyway one key per file is far easier than the stuff we mentioned earlier.

Code:
 awk  -F'|'  '{ file=sprintf("%s.txt", $1); print $0 > file }' filename

creates files named 1.txt, 2.txt, and so on. Plus the input file did not really need to be sorted in any way to do the split.
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

FILE SETS in unix

Hi all, Pls. let me know whether there is any concept called "FILE SETS" in unix? Because, I am using ETL tool DataStage which creates FILE SETS. While I am able to view the data of such a file set in the tool, the "cat" command on this FILESET lists only the Metadata and not the data content... (2 Replies)
Discussion started by: Aparna_A
2 Replies

2. AIX

IP Security file sets

hello, we are implementing ip security on several of our aix 5.2-09 boxes and i am unable to locate the prerequisite file sets. does anyone know where i can find these? i have the original 5.2 cd's but these file sets are not on any of the cd's. Any thoughts or suggestions? (3 Replies)
Discussion started by: zuessh
3 Replies

3. Virtualization and Cloud Computing

Clouds (Partially Order Sets) - Streams (Linearly Ordered Sets) - Part 2

timbass Sat, 28 Jul 2007 10:07:53 +0000 Originally posted in Yahoo! CEP-Interest Here is my follow-up note on posets (partially ordered sets) and tosets (totally or linearly ordered sets) as background set theory for event processing, and in particular CEP and ESP. In my last note, we... (0 Replies)
Discussion started by: Linux Bot
0 Replies

4. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

5. Shell Programming and Scripting

sort split merge -u unique

Hi, this is about sorting a very large file (like 10 gb) to keep lines with unique entries across SOME of the columns. The line originally looked like this: sort -u -k2,2 -k3,3n -k4,4n -k5,5n -k6,6n file_unsorted > file_sorted please note the -u flag. The problem is that this single... (4 Replies)
Discussion started by: jbr950
4 Replies

6. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

7. Shell Programming and Scripting

Identifying dupes within a database and creating unique sub-sets

Hello, I have a database of name variants with the following structure: variant=variant=variant The number of variants can be as many as thirty to forty. Since the database is quite large (at present around 60,000 lines) duplicate sets of variants creep in. Thus John=Johann=Jon and... (2 Replies)
Discussion started by: gimley
2 Replies

8. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

9. UNIX for Beginners Questions & Answers

Split into multiple files by using Unique columns in a UNIX file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|1|1 123|12|5679|BCDEFG|34CD56|2019-07-10|2019-07-10|896.7200|1|2... (3 Replies)
Discussion started by: RVSP
3 Replies
PREZIP-BIN(1)						 Aspell Abbreviated User's Manual					     PREZIP-BIN(1)

NAME
prezip-bin - prefix zip delta word list compressor/decompressor SYNOPSIS
prezip-bin [ -V | -d | -z ] DESCRIPTION
prezip-bin compresses/decompresses sorted word lists from standard input to standard output. Prezip-bin is similar to word-list-compress(1) but it allows a larger character set of {0x00...0x09, 0x0B, 0x0C, 0x0E...0xFF} and multi-words larger than 255 characters in length. It can also decompress word-list-compress(1) compatible files. COMMANDS
Prezip-bin accepts only one of these commands. -V Display prezip-bin version number to standard output. -d Read a compressed word list from standard input and decompress it to standard output. This can be a word-list-compress(1) or a prezip-bin compressed file. -z Read a binary word list from standard input and compress it to standard output. EXAMPLES
prezip-bin -d <wordlist.cwl >wordlist.txt Decompress file wordlist.cwl to text file wordlist.txt prezip-bin -z <wordlist.txt >wordlist.pz 2>errors.txt Compress wordlist.txt to binary file wordlist.pz and send any error messages to a text file named errors.txt LC_COLLATE=C sort -u <wordlist.txt | prezip-bin -z >wordlist.pz Sort a word list, then pipe it to prezip-bin to create a compressed binary wordlist.pz file. prezip-bin -d <words.pz | aspell create master ./words.rws Decompress a wordlist, then pipe it to aspell(1) to create a spelling list. Please check the aspell(1) info manual for proper usage and options. TIPS
Prezip-bin is best used with sorted word list type files. It is not a general purpose compression program since resulting files may actu- ally increase in size. Unlike word-list-compress(1) if your word list has leading or trailing blank spaces for formatting purposes, you should remove them first before you compress your list using prezip-bin -z , otherwise those spaces will be included in the compressed binary output. DIAGNOSTICS
Prezip-bin normally exits with a return code of 0. If it encounters an error, a message is sent to standard error output (stderr), and prezip-bin exits with a non-zero return value. Error messages are listed below: (display help/usage message) Unknown command given on the command line so prezip-bin displays a usage message to standard error output. unknown format The input file appears not to be an expected format, or may possibly be a more advanced format. The output file will be empty. corrupt input This is only for the decompression command -d. The input file appeared to be of a correct format, but something appears wrong now. There may be some valid data in output, but due to input corruption, the rest of the file can not be completed. unexpected EOF The input file appeared okay but ended sooner than expected, therefore the output file is not complete. SEE ALSO
aspell(1), run-with-aspell(1), word-list-compress(1) Aspell is fully documented in its Texinfo manual. See the `aspell' entry in info for more complete documentation. REPORTING BUGS
For help, see the Aspell homepage at <http://aspell.net>. Send bug reports/comments to the Aspell user list at the above address. AUTHOR
This info page was written by Jose Da Silva <digital@joescat.com>. prezip-bin-0.1.2 2005-09-30 PREZIP-BIN(1)
All times are GMT -4. The time now is 08:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy