Sponsored Content
Top Forums UNIX for Advanced & Expert Users Need optimized awk/perl/shell to give the statistics for the Large delimited file Post 303023382 by neutronscott on Friday 14th of September 2018 11:54:42 PM
Old 09-15-2018
something like:

Code:
awk -F\| '!a[$1]++ { print $1 > "dis_col1.txt"; } !b[$2]++ { print $2 > "dis_col2.txt"; } END { print NR; }' file

This User Gave Thanks to neutronscott For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Could someone give me an example of awk accessing array defined in Korn Shell?

As per title and much apprecieated! (2 Replies)
Discussion started by: biglau
2 Replies

2. UNIX for Dummies Questions & Answers

Trim String in 3rd Column in Tab Delimited File...SED/PERL/AWK?

Hey Everybody, I am having much trouble figuring this out, as I am not really a programmer..:mad: Datafile.txt Column0 Column1 Column2 ABC DEF xxxGHI I am running using WGET on a cronjob to grab a datafile, but I need to cut the first three characters from... (6 Replies)
Discussion started by: rickdini
6 Replies

3. Shell Programming and Scripting

Large pipe delimited file that I need to add CR/LF every n fields

I have a large flat file with variable length fields that are pipe delimited. The file has no new line or CR/LF characters to indicate a new record. I need to parse the file and after some number of fields, I need to insert a CR/LF to start the next record. Input file ... (2 Replies)
Discussion started by: clintrpeterson
2 Replies

4. Shell Programming and Scripting

Extracting a portion of data from a very large tab delimited text file

Hi All I wanted to know how to effectively delete some columns in a large tab delimited file. I have a file that contains 5 columns and almost 100,000 rows 3456 f g t t 3456 g h 456 f h 4567 f g h z 345 f g 567 h j k lThis is a very large data file and tab delimited. I need... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

5. Shell Programming and Scripting

Script Optimization - large delimited file, for loop with many greps

Since there are approximately 75K gsfiles and hundreds of stfiles per gsfile, this script can take hours. How can I rewrite this script, so that it's much faster? I'm not as familiar with perl but I'm open to all suggestions. ls file.list>$split for gsfile in `cat $split`; do csplit... (17 Replies)
Discussion started by: verge
17 Replies

6. Shell Programming and Scripting

Awk getting statistics of a grid file,

Hi , I have the following file which is basically a grid (has more than 100000 rows) LLL1 PPP1 LLL1 PPP2 LLL1 PPP3 ............... LLL1 5500 ..... LLL2 PPP1 LLL2 PPP2 LLL2 PPP3 ............... LLL1 5500 ..... L100 PPP1 L100 PPP2 L100 PPP3 ............... 2100 5500... (6 Replies)
Discussion started by: alex2005
6 Replies

7. Shell Programming and Scripting

awk read one delimited file, search another delimited file

Hello folks, I have another doozy. I have two files. The first file has four fields in it. These four fields map to different locations in my second file. What I want to do is read the master file (file 2 - 23 fields) and compare each line against each record in file 1. If I get a match in all four... (4 Replies)
Discussion started by: dagamier
4 Replies

8. Shell Programming and Scripting

Removing dupes within 2 delimited areas in a large dictionary file

Hello, I have a very large dictionary file which is in text format and which contains a large number of sub-sections. Each sub-section starts with the following header : #DATA #VALID 1 and ends with a footer as shown below #END The data between the Header and the Footer consists of... (6 Replies)
Discussion started by: gimley
6 Replies

9. Shell Programming and Scripting

Perl script give answers by file

Hi, I am new in perl. I am running a perl installation script, its asking for paths and so many inputs. Can we provide that info by any file. so i can avoid the interactive installation. (2 Replies)
Discussion started by: Priy
2 Replies
vxusertemplate(1M)														vxusertemplate(1M)

NAME
vxusertemplate - create and administer user templates SYNOPSIS
vxusertemplate -d file [-f] create [template [template ...]] vxusertemplate [-f] create template_name [description=description] [rules=rules] [capability=capability(var=value,...)[,capability...]] vxusertemplate delete template vxusertemplate help [keywords | options | attributes] vxusertemplate list vxusertemplate print template [template ... ] DESCRIPTION
The vxusertemplate utility provides a command line interface for the creation and administration of user-defined or user templates that may be used with the Veritas Intelligent Storage Provisioning (ISP) feature of Veritas Volume Manager (VxVM). The operations that can be performed by vxusertemplate are selected by specifying the appropriate keyword on the command line. See the KEY- WORDS section for a description of the available operations. KEYWORDS
create Creates one or more user templates, whose definitions can be read from a definitions file or entered on the command line. delete Deletes a user template. help Displays information on usage, keywords, options and attributes. list Lists all user templates. print Prints detailed information about one or more user templates. OPTIONS
-d file Specifies a file that contains user template definitions -f Forces an operation that is not usually allowed by vxusertemplate. This option must be specified if the user template definition differs from the existing definition in the configuration database. EXAMPLES
Create the user templates ut1, ut2 and ut3 from the definitions in the file ut1.txt: vxusertemplate -d ut1.txt create ut1,ut2,ut3 Delete the user template myreliabletemplate: vxusertemplate delete myreliabletemplate List all user templates: vxusertemplate list Print details of the user template mytemplate: vxusertemplate print mytemplate EXIT STATUS
The vxusertemplate utility exits with a non-zero status if the attempted operation fails. A non-zero exit code is not a complete indicator of the problems encountered, but rather denotes the first condition that prevented further execution of the utility. SEE ALSO
vxtemplate(1M), vxvoladm(1M) Veritas Storage Foundation Intelligent Storage Provisioning Administrator's Guide VxVM 5.0.31.1 24 Mar 2008 vxusertemplate(1M)
All times are GMT -4. The time now is 07:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy