Sponsored Content
Top Forums Shell Programming and Scripting NAWK array to store lines from huge file Post 302396969 by sharif on Saturday 20th of February 2010 02:18:59 AM
Old 02-20-2010
NAWK array to store lines from huge file

Hi,
I would like to clarify about the NAWK array to store multiple lines from huge file.
The file is having an unique REF.NO, I wants to store the lines (it may be
100+ lines) till I found the new REF.NO.

How can I apply NAWK - arrays for the above?

Rgds,
sharif.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete lines from huge file

I have to delete 1st 7000 lines of a file which is 12GB large. As it is so large, i can't open in vi and delete these lines. Also I found one post here which gave solution using perl, but I don't have perl installed. Also some solutions were redirecting the o/p to a different file and renaming it.... (3 Replies)
Discussion started by: rahulrathod
3 Replies

2. UNIX for Dummies Questions & Answers

How to store the values in a file into an array

Hi, I do have a file and the contents are as follws: 10 20 30 40 50 Now I want to store those values into an array. How can be done this ?? (3 Replies)
Discussion started by: risshanth
3 Replies

3. Shell Programming and Scripting

Prase a file and store and result to an array

Dear all, I have a file having the following formats: ThreadFail=Web1=1234 ThreadFail=Web2=2345 ThreadFail=Web3=12 ConnectionFail=DB1=11 ConnectionFail=DB2=22 The number of lines will be different from every time . How can I parse the file and store the result to an a array inside... (6 Replies)
Discussion started by: youareapkman
6 Replies

4. UNIX for Dummies Questions & Answers

copy and paste certain many lines of huge file in linux

Dear All, I am working with windoes OS but remote a linux machine. I wonder the way to copy an paste some part of a huge file in linux machine. the contain of file like as follow: ... dump annealling all custom 10 anneal_*.dat id type x y z q timestep 0.02 run 200000 Memory... (2 Replies)
Discussion started by: ariesto
2 Replies

5. Shell Programming and Scripting

Most reliable way to store file contents in an array in bash

Hi Guys, I have a file which has numbers in it separated by newlines as follows: 1.113 1.456 0.556 0.021 -0.541 -0.444 I am using the following code to store these in an array in bash: FILE14=data.txt ARRAY14=(`awk '{print}' $FILE14`) (6 Replies)
Discussion started by: npatwardhan
6 Replies

6. Shell Programming and Scripting

Store all the passed arguments in an array and display the array

Hi I want to write a script which store all the parameters passed to the script into an array. Once it is stored I want scan through the array and and delete those files for last month present inside the directory. The files in directory is appneded with YYYY_MM_DD. I want to know how can I... (3 Replies)
Discussion started by: dgmm
3 Replies

7. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

8. Shell Programming and Scripting

how to split a huge file by every 100 lines

into small files. i need to add a head.txt and tail.txt into small files at the begin and end, and give a name as q1.xml q2.xml q3.xml .... thank you very much. (2 Replies)
Discussion started by: dtdt
2 Replies

9. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

10. Shell Programming and Scripting

Store values from a file into an array variable in Shell

Dear All, I have been trying to do a simple task of extracting 2 fields from the file (3 rows) and store it in an array variable. I tried with: #! /bin/bash ch=`cut -f10 tmp.txt` counter=0 for p in $pid do c=${ch} echo "$c ..$counter" counter=$((counter+1))... (2 Replies)
Discussion started by: ezhil01
2 Replies
uniq(1) 						      General Commands Manual							   uniq(1)

Name
       uniq - report repeated lines in a file

Syntax
       uniq [-udc[+n][-n]] [input[output]]

Description
       The  command  reads  the  input	file comparing adjacent lines.	In the normal case, the second and succeeding copies of repeated lines are
       removed; the remainder is written on the output file.  Note that repeated lines must be adjacent in order to be found.  For further  infor-
       mation, see

Options
       The n arguments specify skipping an initial portion of each line in the comparison:

       -n Skips specified number of fields.  A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
	  neighbors.

       +n Skips specified number of characters in addition to fields.  Fields are skipped before characters.

       -c Displays number of repetitions, if any, for each line.

       -d Displays only lines that were repeated.

       -u Displays only unique (nonrepeated) lines.

       If the -u flag is used, just the lines that are not repeated in the original file are output.  The -d option specifies  that  one  copy	of
       just the repeated lines is to be written.  The normal mode output is the union of the -u and -d mode outputs.

       The  -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
       times it occurred.

See Also
       comm(1), sort(1)

																	   uniq(1)
All times are GMT -4. The time now is 10:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy