Sponsored Content
Full Discussion: Complex awk problem
Top Forums Shell Programming and Scripting Complex awk problem Post 302770480 by dietmar13 on Saturday 16th of February 2013 12:18:57 PM
Old 02-16-2013
Complex awk problem

hello,

i have a complex awk problem...

i have two tables, one with a value (0 to 1) and it's corresponding p-value, like this:

1. table:
______________________________
value p-value
... ...
0.254 0.003
0.245 0.005
0.233 0.006
... ...
______________________________

and a second with millions of values (0 to 1), like this:
2. table
______________________________
...
0.252...
0.234...
0.256...
...
______________________________

now I have to map the second list to the first table so that I get for each value the corresponding p-value (i.e. the p-value corresponding to the LOWER value in the 1. table).

expexted output:
______________________________
... ...
0.252... 0.005
0.234... 0.006
9.256... 0.003
... ...
______________________________


one possibility would be to create an indexed array with 1000 instances in this way:
...
a[0.233]=0.006
a[0.234]=0.006
...
a[0.245]=0.005
...

then create a substring of the value of the second list like x.xxx and use this as index for the array.

is there an easier way?

thank you,

dietmar

PS: I have to use AWK, because I already read 7 several gbyte large files an combine these in one table (in minutes) and I have to convert one column on the fly to the corresponding p-values. I think every other programming language will be slower (perhaps except C++, which I can't program).
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Complex Pipeline/Redirection/Regular Expression problem

This is a very hard unix command which I could not perform..... :( ......here it is: The file ~unx122/public/data.txt contains over 18,000 lines of text. Here are the first ten lines of the file: pagination 20657 2740 28416 31090 18145 shiner 1695 2507 9964 14512 13122 cool 29210... (1 Reply)
Discussion started by: netmaster
1 Replies

2. Shell Programming and Scripting

Complex problem about nested for loops

Hey, I'm writing this bash script that will test print me many copies of the same program but with different combos of 4 variables being between 1 and 100. Here's the code: #! /bin/bash x=0 for ((a=1; a < 101; a++)) do for ((b=1; b < 101; b++)) do for ((c=1; c < 101; c++)) do for... (4 Replies)
Discussion started by: Silverlining
4 Replies

3. Shell Programming and Scripting

Complex file count problem

Hi all! I have a question regarding possibilities to do line counts. SEARCH_VAR=TEX rsh $REM_HOST -l $REM_USER "cd $REM_DIR; ls *$SEARCH_VAR* 2> /dev/null" | sort -n | awk 'BEGIN { FS = "-" } ; { print $1"\t"$0 }' Will produce an output on the screen like this: 483 483-SOME-TEXT-1... (1 Reply)
Discussion started by: bbergstrom74
1 Replies

4. IP Networking

Need to solve complex network problem

I have a Red Hat linux server X on a x.x.0.0 network. This machine also has to communicate with another server Y on a network called y.y.0.0 Server X has two network interfaces. eth0 is configured on the x.x.0.0 network and has a default gateway on the x.x.0.0 network. In order to... (4 Replies)
Discussion started by: soliberus
4 Replies

5. Shell Programming and Scripting

Complex use with awk

Hi , I have file named docs.txt The content of the file look like this: DOC disk location Size ======= ===== ============= ========= TXT A /dev/dm-1 10 TXT B /dev/dm-2 10 BIN C ... (3 Replies)
Discussion started by: yoavbe
3 Replies

6. Shell Programming and Scripting

complex Awk Question

Hi, I have a file look likes this : --->start hir Trace file: pudwh_ora_9998.trc Sort options: fchela exeela ***************************************************************count = number of times OCI procedure was executed cpu = cpu time in seconds executing elapsed = elapsed... (3 Replies)
Discussion started by: yoavbe
3 Replies

7. Shell Programming and Scripting

Get values from 2 files - Complex "for loop and if" awk problem

Hi everyone, I've been thinking and trying/changing all day long the below code, maybe some awk expert could help me to fix the for loop I've thought, I think I'm very close to the correct output. file1 is: <boxes content="Grapes and Apples"> <box No.="Box MT. 53"> <quantity... (8 Replies)
Discussion started by: Ophiuchus
8 Replies

8. Shell Programming and Scripting

awk script (complex)

picked this up from another thread. echo 1st_file.csv; nawk -F, 'NR==FNR{a++;next} a{b++} END{for(i in b){if(b-1&&a!=b){print i";\t\t"b}else{print "NEW:"i";\t\t"b} } }' OFS=, 1st_file.csv *.csv | sort -r i need to use the above but with a slight modification.. 1.compare against 3 month... (25 Replies)
Discussion started by: slashbash
25 Replies

9. Shell Programming and Scripting

Help with Complex Awk.

Hi, I have a file. In this file when ever the word "ABC" occurs at position from 25 and 34 I would like to replace the value at postion 100 to 5 for the first 1000 rows only. I have written the following Awk command. nawk 'substr($0,25,9)=="ABC" {print $0}' filename The above command... (4 Replies)
Discussion started by: pinnacle
4 Replies

10. Shell Programming and Scripting

Complex text parsing with speed/performance problem (awk solution?)

I have 1.6 GB (and growing) of files with needed data between the 11th and 34th line (inclusive) of the second column of comma delimited files. There is also a lot of stray white space in the file that needs to be trimmed. They have DOS-like end of lines. I need to transpose the 11th through... (13 Replies)
Discussion started by: Michael Stora
13 Replies
iconv_pc_cyr(5)                                         Standards, Environments, and Macros                                        iconv_pc_cyr(5)

NAME
iconv_pc_cyr - code set conversion tables for Alternative PC Cyrillic DESCRIPTION
The following code set conversions are supported: +---------------------------------------------------------------------+ | Code Set Conversions Supported | +--------------+--------+--------------+--------+---------------------+ | Code |Symbol |Target Code |Symbol | Target Output | +--------------+--------+--------------+--------+---------------------+ |PC Cyrillic |alt |ISO 8859-5 |iso5 | ISO 8859-5 Cyrillic | +--------------+--------+--------------+--------+---------------------+ |PC Cyrillic |alt |KOI8-R |koi8 | KOI8-R | +--------------+--------+--------------+--------+---------------------+ |PC Cyrillic |alt |MS 1251 |win5 | Windows Cyrillic | +--------------+--------+--------------+--------+---------------------+ |PC Cyrillic |alt |Mac Cyrillic |mac | Macintosh Cyrillic | +--------------+--------+--------------+--------+---------------------+ CONVERSIONS
The conversions are performed according to the following tables. All values in the tables are given in octal. PC Cyrillic to ISO 8859-5 For the conversion of PC Cyrillic to ISO 8859-5, all characters not in the following table are mapped unchanged. +-----------------------------------------------------------------+ | | Conversions|Performed | | | PC Cyrillic | ISO 8859-5 | PC Cyrillic | ISO 8859-5 | |24 | 4 |231 |311 | |200 | 260 |232 |312 | |201 | 261 |233 |313 | |202 | 262 |234 |314 | |203 | 263 |235 |315 | |204 | 264 |236 |316 | |205 | 265 |237 |317 | |206 | 266 |240 |320 | |207 | 267 |241 |321 | |210 | 270 |242 |322 | |211 | 271 |243 |323 | |212 | 272 |244 |324 | |213 | 273 |245 |325 | |214 | 274 |246 |326 | |215 | 275 |247 |327 | |216 | 276 |250 |330 | |217 | 277 |251 |331 | |220 | 300 |252 |332 | |221 | 301 |253 |333 | |222 | 302 |254 |334 | |223 | 303 |255 |335 | |224 | 304 |256 |336 | |225 | 305 |257 |337 | |226 | 306 |260-337 |255 | |227 | 307 |360 |241 | |230 | 310 |362-376 |255 | +---------------+----------------+----------------+---------------+ PC Cyrillic to KOI8-R For the conversion of PC Cyrillic to KOI8-R, all characters not in the following table are mapped unchanged. +-----------------------------------------------------------------+ | | Conversions|Performed | | | PC Cyrillic | KOI8-R | PC Cyrillic | KOI8-R | |24 | 4 |242 |327 | |200 | 341 |243 |307 | |201 | 342 |244 |304 | |202 | 367 |245 |305 | |203 | 347 |246 |326 | |204 | 344 |247 |332 | |205 | 345 |250 |311 | |206 | 366 |251 |312 | |207 | 372 |252 |313 | |210 | 351 |253 |314 | |211 | 352 |254 |315 | |212 | 353 |255 |316 | |213 | 354 |256 |317 | |214 | 355 |257 |320 | |215 | 356 |260-337 |255 | |216 | 357 |340 |322 | |217 | 360 |341 |323 | |220 | 362 |342 |324 | |221 | 363 |343 |325 | |222 | 364 |344 |306 | |223 | 365 |345 |310 | |224 | 346 |346 |303 | |225 | 350 |347 |336 | |226 | 343 |350 |333 | |227 | 376 |351 |335 | |230 | 373 |352 |337 | |231 | 375 |353 |331 | |232 | 377 |354 |330 | |233 | 371 |355 |334 | |234 | 370 |356 |300 | |235 | 374 |357 |321 | |236 | 340 |360 |263 | |237 | 361 |361 |243 | |240 | 301 |362-376 |255 | |241 | 302 | | | +---------------+----------------+----------------+---------------+ PC Cyrillic to MS 1251 For the conversion of PC Cyrillic to MS 1251, all characters not in the following table are mapped unchanged. +-----------------------------------------------------------------+ | | Conversions|Performed | | | PC Cyrillic | MS 1251 | PC Cyrillic | MS 1251 | |24 | 4 |242 |342 | |200 | 300 |243 |343 | |201 | 301 |244 |344 | |202 | 302 |245 |345 | |203 | 303 |246 |346 | |204 | 304 |247 |347 | |205 | 305 |250 |350 | |206 | 306 |251 |351 | |207 | 307 |252 |352 | |210 | 310 |253 |353 | |211 | 311 |254 |354 | |212 | 312 |255 |355 | |213 | 313 |256 |356 | |214 | 314 |257 |357 | |215 | 315 |260-337 |210 | |216 | 316 |340 |360 | |217 | 317 |341 |361 | |220 | 320 |342 |362 | |221 | 321 |343 |363 | |222 | 322 |344 |364 | |223 | 323 |345 |365 | |224 | 324 |346 |366 | |225 | 325 |347 |367 | |226 | 326 |350 |370 | |227 | 327 |351 |371 | |230 | 330 |352 |372 | |231 | 331 |353 |373 | |232 | 332 |354 |374 | |233 | 333 |355 |375 | |234 | 334 |356 |376 | |235 | 335 |357 |377 | |236 | 336 |360 |250 | |237 | 337 |361 |270 | |240 | 340 |362-376 |210 | |241 | 341 | | | +---------------+----------------+----------------+---------------+ PC Cyrillic to Mac Cyrillic For the conversion of PC Cyrillic to Mac Cyrillic, all characters not in the following table are mapped unchanged. +-----------------------------------------------------------------+ | | Conversions|Performed | | | PC Cyrillic | Mac Cyrillic | PC Cyrillic | Mac Cyrillic | |24 | 4 |341 |361 | |240 | 340 |342 |362 | |241 | 341 |343 |363 | |242 | 342 |344 |364 | |243 | 343 |345 |365 | |244 | 344 |346 |366 | |245 | 345 |347 |367 | |246 | 346 |350 |370 | |247 | 347 |351 |371 | |250 | 350 |352 |372 | |251 | 351 |353 |373 | |252 | 352 |354 |374 | |253 | 353 |355 |375 | |254 | 354 |356 |376 | |255 | 355 |357 |337 | |256 | 356 |360 |335 | |257 | 357 |361 |336 | |260-337 | 40 |362-376 |40 | |340 | 360 | | | +---------------+----------------+----------------+---------------+ FILES
/usr/lib/iconv/*.so conversion modules /usr/lib/iconv/*.t conversion tables /usr/lib/iconv/iconv_data list of conversions supported by conversion tables SEE ALSO
iconv(1), iconv(3C), iconv(5) SunOS 5.10 18 Apr 1997 iconv_pc_cyr(5)
All times are GMT -4. The time now is 11:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy