Sponsored Content
Full Discussion: Split file based on field
Top Forums Shell Programming and Scripting Split file based on field Post 302314143 by s_adu on Thursday 7th of May 2009 01:06:53 PM
Old 05-07-2009
Yes

I have records like

a|b|b|c|c|dsda|ada|123
a|b1|b|c1|c|dsda|ada|223
a|b1|b|c1|c|dsda|ada|223
a|b|b|c|c|dsda|ada|423
a|b|b|c|c|dsda|ada|1234
a|b|b|c|c|dsda|ada|123

I require

123
a|b|b|c|c|dsda|ada|123
a|b|b|c|c|dsda|ada|123

223
a|b1|b|c1|c|dsda|ada|223
a|b1|b|c1|c|dsda|ada|223

1234
a|b|b|c|c|dsda|ada|1234

423
a|b|b|c|c|dsda|ada|423


Actually awk -F"|" '{ print > $NF }' filename1 is doing the job. My input file has 2.6 million records. It created some 1000 files and then giving a strange error

Please advice me

Adarsh
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Please Help:Need to Split the file into mutliple files depends on the KEY field value

Hi Gurus, I am new to UNIX(HP). my requirmnet is File needs to needs to be split into multiple files dependa on one key value. example 1 abc 333 us 2 bbc 444 ch 5 nnn 333 kk 7 hhh 555 ll 3 hhh 333 hh now the requirment is line with 333 should be copied into test1.txt and... (14 Replies)
Discussion started by: arund_01
14 Replies

2. Shell Programming and Scripting

split large file based on field criteria

I have a file containing date/time sorted data of the form ... 2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1 2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1 2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0 2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies

3. Shell Programming and Scripting

Split file when the key field change !

Hello, I have the following example data file: Rv.Global_Sk,1077.160523,D,16/09/2011 Rv.Global_Sk,1077.08098,D,17/09/2011 Rv.Global_Sk,1077.001445,D,18/09/2011 Rv.Global_Sk,1072.660733,D,19/09/2011 Rv.Global_Sk,1070.381557,D,20/09/2011 Rv.Global_Sk,1071.971747,D,21/09/2011... (4 Replies)
Discussion started by: csierra
4 Replies

4. Shell Programming and Scripting

awk to split one field and print the last two fields within the split part.

Hello; I have a file consists of 4 columns separated by tab. The problem is the third fields. Some of the them are very long but can be split by the vertical bar "|". Also some of them do not contain the string "UniProt", but I could ignore it at this moment, and sort the file afterwards. Here is... (5 Replies)
Discussion started by: yifangt
5 Replies

5. Shell Programming and Scripting

Split a file into multiple files based on field value

Hi, I've one requirement. I have to split one comma delimited file into multiple files based on one of the column values. How can I achieve this Unix Here is the sample data. In this case I have split the files based on date column(c4) Input file c1,c2,c3,c4,c5... (1 Reply)
Discussion started by: manasvi24
1 Replies

6. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

7. Shell Programming and Scripting

perl script to split the text file after every 4th field

I had a text file(comma seperated values) which contains as below 196237,ram,25-May-06,ram.kiran@xyz.com,204183,Pavan,4-Jun-07,Pavan.Desai@xyz.com,237107,ram Chandra,15-Mar-10,ram.krishna@xyz.com ... (3 Replies)
Discussion started by: giridhar276
3 Replies

8. Shell Programming and Scripting

Split the file based on column

Hi, I have a file sample_1.txt (300k rows) which has data like below: * Also each record is around 64k bytes 11|1|abc|102553|125589|64k bytes of data 10|2|def|123452|123356|...... 13|2|geh|144351|121123|... 25|4|fgh|165250|118890|.. 14|1|abc|186149|116657|......... (6 Replies)
Discussion started by: sol_nov
6 Replies

9. UNIX for Dummies Questions & Answers

Split a file into parts only if the first field is different

Hi, I have a file like this: aaa 123 aaa 223 aaa 225 bbb 332 bbb 423 bbb 6755 bbb 324 ccc 112 ccc 234 ccc 897 Which I need to split into several files, something like split -l 3 but the way that the lines with the same names would only go into one file: (7 Replies)
Discussion started by: coppuca
7 Replies

10. Shell Programming and Scripting

Split file based on a column/field value

Hi All, I have a requirement to split file into 2 sets of file. Below is a sample data of the file AU;PTN;24EX;25-AUG-14;AU;123;SE;123;Test NN;;;;ASD; AU;PTN;24EX;25-AUG-14;AU;456;SE;456;Test NN;;;;ASD; AU;PTN;24EX;25-AUG-14;AU;147;SE;147;Test NN;;;;ASD;... (6 Replies)
Discussion started by: galaxy_rocky
6 Replies
ADA(4)							   BSD Kernel Interfaces Manual 						    ADA(4)

NAME
ada -- ATA Direct Access device driver SYNOPSIS
device ada DESCRIPTION
The ada driver provides support for direct access devices, implementing the ATA command protocol, that are attached to the system through a host adapter supported by the CAM subsystem. The host adapter must also be separately configured into the system before an ATA direct access device can be configured. COMMAND QUEUING
Command queueing allows the device to process multiple transactions concurrently, often re-ordering them to reduce the number and length of seeks. ATA defines two types of queueing: TCQ (Tagged Command Queueing, PATA legacy) and NCQ (Native Command Queueing, SATA). The ada device driver takes full advantage of NCQ, when supported. To ensure that transactions to distant parts of the media, which may be deferred indefinitely by servicing requests closer to the current head position, are completed in a timely fashion, an ordered transaction is sent every 7 seconds during continuous device operation. CACHE EFFECTS
Many direct access devices are equipped with read and/or write caches. Parameters affecting the device's cache are reported in device IDEN- TIFY data and can be examined and modified via the camcontrol(8) utility. The read cache is used to store data from device-initiated read ahead operations as well as frequently used data. The read cache is trans- parent to the user and can be enabled without any adverse effect. Most devices with a read cache come from the factory with it enabled. The write cache can greatly decrease the latency of write operations and allows the device to reorganize writes to increase efficiency and performance. This performance gain comes at a price. Should the device lose power while its cache contains uncommitted write operations, these writes will be lost. The effect of a loss of write transactions on a file system is non-deterministic and can cause corruption. Most devices age write transactions to limit the vulnerability to a few transactions recently reported as complete, but it is nonetheless recom- mended that systems with write cache enabled devices reside on an Uninterruptible Power Supply (UPS). The ada device driver ensures that the cache and media are synchronized upon final close of the device or an unexpected shutdown (panic) event. This ensures that it is safe to disconnect power once the operating system has reported that it has halted. SYSCTL VARIABLES
The following variables are available as both sysctl(8) variables and loader(8) tunables: kern.cam.ada.retry_count This variable determines how many times the ada driver will retry a READ or WRITE command. This does not affect the number of retries used during probe time or for the ada driver dump routine. This value currently defaults to 4. kern.cam.ada.default_timeout This variable determines how long the ada driver will wait before timing out an outstanding command. The units for this value are sec- onds, and the default is currently 30 seconds. kern.cam.ada.spindown_shutdown This variable determines whether to spin-down disks when shutting down. Set to 1 to enable spin-down, 0 to disable. The default is cur- rently enabled. FILES
/dev/ada* ATA device nodes SEE ALSO
ad(4), ahci(4), cam(4), da(4), siis(4) HISTORY
The ada driver first appeared in FreeBSD 8.0. AUTHORS
Alexander Motin <mav@FreeBSD.org> BSD
October 24, 2010 BSD
All times are GMT -4. The time now is 06:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy