Sponsored Content
Top Forums Shell Programming and Scripting Lookup on large file based on a temp file Post 302640067 by ygemici on Monday 14th of May 2012 04:44:04 AM
Old 05-14-2012
Code:
# awk -F'|' 'FNR==NR{a[$1]}($4 in a){A[$4,x]++}END{for(i in A)print i FS A[i]}' base lookup
abc-001|3
bcd-001|2
cde-001|4

This User Gave Thanks to ygemici For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Clueless about how to lookup and reverse lookup IP addresses under a file!!.pls help

Write a quick shell snippet to find all of the IPV4 IP addresses in any and all of the files under /var/lib/output/*, ignoring whatever else may be in those files. Perform a reverse lookup on each, and format the output neatly, like "IP=192.168.0.1, ... (0 Replies)
Discussion started by: choco4202002
0 Replies

2. Shell Programming and Scripting

SED based on file lookup

Newb here trying to figure this one out. :confused: I am trying to create a SED (or some other idea) line that will replace the data field if the original text is seen in a separate text file. The lookup file would be line delimted. For example: sed 's/<if in file>/YES/' File structure:... (3 Replies)
Discussion started by: sdlennon
3 Replies

3. Shell Programming and Scripting

split large file based on field criteria

I have a file containing date/time sorted data of the form ... 2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1 2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1 2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0 2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies

4. Shell Programming and Scripting

Split large file based on last digit from a column

Hello, What's the best way to split a large into multiple files based on the last digit in the first column. input file: f 2738483300000x0y03772748378831x1y13478378358383x2y23743878383802x3y33787828282820x4y43748838383881x5y5 Desired Output: f0 3738483300000x0y03787828282820x4y4 f1... (9 Replies)
Discussion started by: alain.kazan
9 Replies

5. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies

6. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

7. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

8. Shell Programming and Scripting

Splitting large file and renaming based on field

I am trying to update an older program on a small cluster. It uses individual files to send jobs to each node. However the newer database comes as one large file, containing over 10,000 records. I therefore need to split this file. It looks like this: HMMER3/b NAME 1-cysPrx_C ACC ... (2 Replies)
Discussion started by: fozrun
2 Replies

9. Shell Programming and Scripting

Help with Splitting a Large XML file based on size AND tags

Hi All, This is my first post here. Hoping to share and gain knowledge from this great forum !!!! I've scanned this forum before posting my problem here, but I'm afraid I couldn't find any thread that addresses this exact problem. I'm trying to split a large XML file (with multiple tag... (7 Replies)
Discussion started by: Aviktheory11
7 Replies

10. Shell Programming and Scripting

Delete the records in file based on lookup file.

Hi I have two files one.txt and two.txt one.txt 123 324 456 235 456 two txt abc one 000 123 abc abc one 000 456 abc abc one 000 122 abc abc one 000 111 abc My question here is, the records which are present in one.txt has to deleted in second file two.txt my output result... (2 Replies)
Discussion started by: Ganesh L
2 Replies
NIFTI_STATS(1)							   User Commands						    NIFTI_STATS(1)

NAME
nifti_stats - compute NIfTI statistical functions SYNOPSIS
nifti_stats [-q|-d|-1|-z] value CODE [p1 p2 p3] DESCRIPTION
nifti_stats supports several distributions (normal, uniform, logistic, chi, etc.) and calculates density or cummulative distribution func- tion (and many more). Values are printed to stdout and can be piped to other tools. value can be a single number or in the form bot:top:step. default ==> output p = Prob(statistic < val). -q ==> output is 1-p. -d ==> output is density. -1 ==> output is x such that Prob(statistic < x) = val. -z ==> output is z such that Normal cdf(z) = p(val). -h ==> output is z such that 1/2-Normal cdf(z) = p(val). Allowable CODEs: CORREL, TTEST, FTEST, ZSCORE, CHISQ, BETA, BINOM, GAMMA, POISSON, NORMAL, FTEST_NONC, CHISQ_NONC, LOGISTIC, LAPLACE, UNIFORM, TTEST_NONC, WEIBULL, CHI, INVGAUSS, EXTVAL, PVAL, LOGPVAL and LOG10PVAL Following CODE are distributional parameters, as needed. Results are written to stdout, 1 number per output line. EXAMPLE
Piping output into AFNI program 1dplot: nifti_stats -d 0:4:.001 INVGAUSS 1 3 | 1dplot -dx 0.001 -stdin SEE ALSO
libnifti(1), nifti_tool(1), nifti1_test(1). Homepage: http://niftilib.sourceforge.net AUTHOR
RW Cox - SSCC/NIMH/NIH/DHHS/USA/EARTH - March 2004 This manual page was written by Michael Hanke <michael.hanke@gmail.com>, for the Debian project (but may be used by others). nifti_stats September 2007 NIFTI_STATS(1)
All times are GMT -4. The time now is 07:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy