removing line and duplicate line

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

2. Shell Programming and Scripting

Honey, I broke awk! (duplicate line removal in 30M line 3.7GB csv file)

I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code: awk...

3. Shell Programming and Scripting

Duplicate third column to every line

Dear All, I have file input like this: INP901 5173 4114 INP902 5227 INP903 5284 INP904 5346 INP905 5400 INP906 5456 INP907 5511 INP908 5572 INP909 5622 INP910 5678 INP911 5739 INP912 5796 INP913 5845 INP914 5910 INP915 5965

4. UNIX for Dummies Questions & Answers

Duplicate a line n times

I want a command equivalent to sed 'p;p' file My input file contains 1 2 3 If i want to input file like this 1 1 1 2 2 2 3 3 3 I use

5. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from...

6. Shell Programming and Scripting

Removing a line IF the next line contains string

So, I've been working on a project which takes layer 7 metadata from pcap dumps and archives it. However, there is a lot of dataless information that I don't want in my output. I know of ways to produce the output I want from the input file below, but I want a method of doing this, regardless of...

7. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ...

8. Shell Programming and Scripting

Duplicate Line Report per Section

I've been working on a script (/bin/sh) in which I have requested and received help here (in which I am very grateful for!). The client has modified their requirements (a tad), so without messing up the script to much, I come once again for assistance. Here are the file.dat contents: ABC1...

9. Shell Programming and Scripting

remove duplicate words in a line

Hi, Please help! I have a file having duplicate words in some line and I want to remove the duplicate words. The order of the words in the output file doesn't matter. INPUT_FILE pink_kite red_pen ball pink_kite ball yellow_flower white no white no cloud nine_pen pink cloud pink nine_pen...

10. UNIX for Dummies Questions & Answers

Remove Duplicate line

Hi, I have a scenario here where I have created a flatfile with the below mentioned information. File as you can see is dispalyed in three columns 1st column is FileNameString 2nd column is Report_Name (this has spaces) 3rd column is Flag Result file needed is, removal of duplicate...

LEARN ABOUT NETBSD

lpq

LPQ(1)							    BSD General Commands Manual 						    LPQ(1)

NAME

     lpq -- spool queue examination program

SYNOPSIS

     lpq [-al] [-P printer] [-w maxwait] [job# ...] [user ...]

DESCRIPTION

     lpq examines the spooling area used by lpd(8) for printing files on the line printer, and reports the status of the specified jobs or all
     jobs associated with a user.  lpq invoked without any arguments reports on any jobs currently in the queue.

     Options:

     -P      Specify a particular printer, otherwise the default line printer is used (or the value of the PRINTER variable in the environment).
	     All other arguments supplied are interpreted as user names or job numbers to filter out only those jobs of interest.

     -l      Information about each of the files comprising the job entry is printed.  Normally, only as much information as will fit on one line
	     is displayed.

     -a      Report on the local queues for all printers, rather than just the specified printer.

     -w maxwait
	     Specify the maximum time to wait in seconds for remote responses.	The default is 300 seconds or 5 minutes.

     For each job submitted (i.e. invocation of lpr(1)) lpq reports the user's name, current rank in the queue, the names of files comprising the
     job, the job identifier (a number which may be supplied to lprm(1) for removing a specific job), and the total size in bytes.  Job ordering
     is dependent on the algorithm used to scan the spooling directory and is supposed to be FIFO (First in First Out).  File names comprising a
     job may be unavailable (when lpr(1) is used as a sink in a pipeline) in which case the file is indicated as ``(standard input)''.

     If lpq warns that there is no daemon present (i.e. due to some malfunction), the lpc(8) command can be used to restart the printer daemon.

ENVIRONMENT

     If the following environment variable exists, it is used by lpq:

     PRINTER  Specifies an alternative default printer.

FILES

     /etc/printcap	       To determine printer characteristics.
     /var/spool/output/*       The spooling directory, as determined from printcap.
     /var/spool/output/*/cf*   Control files specifying jobs.
     /var/spool/output/*/lock  The lock file to obtain the currently active job.

DIAGNOSTICS

     Unable to open various files.  The lock file being malformed.  Garbage files when there is no daemon active, but files in the spooling direc-
     tory.

SEE ALSO

     lpr(1), lprm(1), lpc(8), lpd(8)

HISTORY

     lpq appeared in 3BSD.

BUGS

     Due to the dynamic nature of the information in the spooling directory lpq may report unreliably.	Output formatting is sensitive to the line
     length of the terminal; this can results in widely spaced columns.

BSD
								  April 28, 1995							       BSD

UNIX for Dummies Questions & Answers