Sponsored Content
Top Forums Shell Programming and Scripting search a number in very very huge amount of data Post 302578738 by vsachan on Friday 2nd of December 2011 08:09:28 AM
Old 12-02-2011
The file format is like :
Code:
00.87.123.45|54999999999|98765|-|[15/NOV/2011:20:02:08 |"www.unix.com/"|GTR|1.1.1|UU|654|0012|0|[TTTTTTTT]|0432|text/html|SDEWRTRERERERERER==|1.4.5.6|fkjgkjfgfg|-|-|-|-|-|-|-|-|-|-|-|2|123456789|0|1||123,45,7654,76123|345|67|45654654645645645|54.67.4.345|323423423|4567|56098|-|

I used awk command to search.I already know that the serached pattern occurs in 2nd field.So compare the string only with 2nd field.but when I am using -v option to pass a variable to awk command but it is not searching the variable in the file, while if I hardcode the value then it is searching the variable.Is there any mistake in the below code?

script:
-----
Code:
variable=$1
for i in `ls *`
do
gunzip -c $i | awk -F"|" -v var="$variable" '$2~/variable/' >> output.txt &
done
 
 
 
./script 999999999


Last edited by Franklin52; 12-03-2011 at 11:49 AM.. Reason: Please use code tags for data and code samples, thank you
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies

3. UNIX for Advanced & Expert Users

Best way to search for patterns in huge text files

I have the following situation: a text file with 50000 string patterns: abc2344536 gvk6575556 klo6575556 .... and 3 text files each with more than 1 million lines: ... 000000 abc2344536 46575 0000 000000 abc2344536 46575 4444 000000 abc2344555 46575 1234 ... I... (8 Replies)
Discussion started by: andy2000
8 Replies

4. Shell Programming and Scripting

How to delete a huge number of files at a time

I met a problem on HPUX with 64G RAM and 20 CPU. There are 5 million files with file name from file0000001.dat to file9999999.dat, in the same directory, and with some other files with random names. I was trying to remove all the files from file0000001.dat to file9999999.dat at the same time.... (9 Replies)
Discussion started by: lisp21
9 Replies

5. Shell Programming and Scripting

search a string in a huge file

How to search a string which has occured numerous times in a single row. I tried many options, I am facing issue with the file size. Anything I go for, it says it is huge.. File is 82MB. Assume, the file contains the string 'Name' in many places.. Something Like below. ... (5 Replies)
Discussion started by: Muthuraj K
5 Replies

6. AIX

Error while copying huge amount of data in aix

Hi When i copy 300GB of data from one filesystem to the other filesystem in AIX I get the error : tar: 0511-825 The file 'SAPBRD.dat' is too large. The command I used is : # tar -cf - . | (cd /sapbackup ; tar -xf - ) im copying as root The below is my ulimit -a output : ... (3 Replies)
Discussion started by: samsungsamsung
3 Replies

7. Shell Programming and Scripting

Search and replace ---A huge number of files

Hello Friends, I have the below scenario in my current project. Suggest me which tool ( perl,python etc) is best to this scenario. Or should I go for Programming language ( C/Java ).. (1) I will be having a very big file ( information about 200million subscribers will be stored in it ). This... (5 Replies)
Discussion started by: panyam
5 Replies

8. Shell Programming and Scripting

Perl : Large amount of data put into an array

This basic code works. I have a very long list, almost 10000 lines that I am building into the array. Each line has either 2 or 3 fields as shown in the code snippit. The array elements are static (for a few reasons that out of scope of this question) the list has to be "built in". It... (5 Replies)
Discussion started by: sumguy
5 Replies

9. Shell Programming and Scripting

Aggregation of huge data

Hi Friends, I have a file with sample amount data as follows: -89990.3456 8788798.990000128 55109787.20 -12455558989.90876 I need to exclude the '-' symbol in order to treat all values as an absolute one and then I need to sum up.The record count is around 1 million. How... (8 Replies)
Discussion started by: Ravichander
8 Replies

10. Shell Programming and Scripting

How to make awk command faster for large amount of data?

I have nginx web server logs with all requests that were made and I'm filtering them by date and time. Each line has the following structure: 127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br) These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies
Locale::Codes::Script(3)				User Contributed Perl Documentation				  Locale::Codes::Script(3)

NAME
Locale::Codes::Script - standard codes for script identification SYNOPSIS
use Locale::Codes::Script; $script = code2script('phnx'); # 'Phoenician' $code = script2code('Phoenician'); # 'Phnx' $code = script2code('Phoenician', LOCALE_CODE_NUMERIC); # 115 @codes = all_script_codes(); @scripts = all_script_names(); DESCRIPTION
The "Locale::Codes::Script" module provides access to standards codes used for identifying scripts, such as those defined in ISO 15924. Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 15924 four-letter codes will be used. SUPPORTED CODE SETS
There are several different code sets you can use for identifying scripts. A code set may be specified using either a name, or a constant that is automatically exported by this module. For example, the two are equivalent: $script = code2script('phnx','alpha'); $script = code2script('phnx',LOCALE_SCRIPT_ALPHA); The codesets currently supported are: alpha, LOCALE_SCRIPT_ALPHA This is a set of four-letter (capitalized) codes from ISO 15924 such as 'Phnx' for Phoenician. It also includes additions to this set included in the IANA language registry. The Zxxx, Zyyy, and Zzzz codes are not used. This is the default code set. num, LOCALE_SCRIPT_NUMERIC This is a set of three-digit numeric codes from ISO 15924 such as 115 for Phoenician. ROUTINES
code2script ( CODE [,CODESET] ) script2code ( NAME [,CODESET] ) script_code2code ( CODE ,CODESET ,CODESET2 ) all_script_codes ( [CODESET] ) all_script_names ( [CODESET] ) Locale::Codes::Script::rename_script ( CODE ,NEW_NAME [,CODESET] ) Locale::Codes::Script::add_script ( CODE ,NAME [,CODESET] ) Locale::Codes::Script::delete_script ( CODE [,CODESET] ) Locale::Codes::Script::add_script_alias ( NAME ,NEW_NAME ) Locale::Codes::Script::delete_script_alias ( NAME ) Locale::Codes::Script::rename_script_code ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::Script::add_script_code_alias ( CODE ,NEW_CODE [,CODESET] ) Locale::Codes::Script::delete_script_code_alias ( CODE [,CODESET] ) These routines are all documented in the Locale::Codes::API man page. SEE ALSO
Locale::Codes The Locale-Codes distribution. Locale::Codes::API The list of functions supported by this module. http://www.unicode.org/iso15924/ Home page for ISO 15924. http://www.iana.org/assignments/language-subtag-registry The IANA language subtag registry. AUTHOR
See Locale::Codes for full author history. Currently maintained by Sullivan Beck (sbeck@cpan.org). COPYRIGHT
Copyright (c) 1997-2001 Canon Research Centre Europe (CRE). Copyright (c) 2001-2010 Neil Bowers Copyright (c) 2010-2013 Sullivan Beck This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.16.3 2013-02-27 Locale::Codes::Script(3)
All times are GMT -4. The time now is 09:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy