Sponsored Content
Top Forums Shell Programming and Scripting Finding duplicates in a file excluding specific pattern Post 302819493 by shiva2985 on Tuesday 11th of June 2013 02:53:33 AM
Old 06-11-2013
Finding duplicates in a file excluding specific pattern

I have unix file like below
>newuser
newuser
<hello
hello
newone

I want to find the unique values in the file(excluding <,>),so that the out put should be
>newuser
<hello
newone

can any body tell me what is command to get this new file.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding a specific pattern from thousands of files ????

Hi All, I want to find a specific pattern from approximately 400000 files on solaris platform. Its very heavy for me to grep that pattern to each file individually. Can anybody suggest me some way to search for specific pattern (alpha numeric) from these forty thousand files. Please note that... (6 Replies)
Discussion started by: aarora_98
6 Replies

2. Shell Programming and Scripting

Remove duplicates from File from specific location

How can i remove the duplicate lines from a file, for example sample123456Sample testing123456testing XXXXX131323XXXXX YYYYY423432YYYYY fsdfdsf123456gsdfdsd all the duplicates from column 6-12 , must be deleted. I want to consider the first row, if same comes in the given range i want to... (1 Reply)
Discussion started by: gopikgunda
1 Replies

3. UNIX for Dummies Questions & Answers

removing duplicates of a pattern from a file

hey all, I need some help. I have a text file with names in it. My target is that if a particular pattern exists in that file more than once..then i want to rename all the occurences of that pattern by alternate patterns.. for e.g if i have PATTERN occuring 5 times then i want to... (3 Replies)
Discussion started by: ashisharora
3 Replies

4. Shell Programming and Scripting

Finding 4 current files having specific File Name pattern

Hi All, I am trying to find 4 latest files inside one folder having following File Name pattern and store them into 4 different variables and then use for processing in my shell script. File name is fixed length. 1) Each file starts with = ABCJmdmfbsjop letters + 7 Digit Number... (6 Replies)
Discussion started by: lancesunny
6 Replies

5. UNIX for Dummies Questions & Answers

Finding new file, but excluding directory..

hi, I need to find files that have been created less than 3 days ago. However, I need to only search specific directories. I've searched about the net and found some useful commands such as : find . -type d -name 'dir_to_exclude' -prune -o -print -mtime -3 however I cannot get it... (2 Replies)
Discussion started by: horhif
2 Replies

6. Shell Programming and Scripting

Finding the pattern and replacing the pattern inside the file

i have little challenge, help me out.i have a file where i have a value declared and and i have to replace the value when called. for example i have the value for abc and ccc. now i have to substitute the value of value abc and ccc in the place of them. Input File: go to &abc=ddd; if... (16 Replies)
Discussion started by: saaisiva
16 Replies

7. Shell Programming and Scripting

Search for duplicates and delete but remain the first one based on a specific pattern

Hi all, I have been trying to delete duplicates based on a certain pattern but failed to make it works. There are more than 1 pattern which are duplicated but i just want to remove 1 pattern only and remain the rest. I cannot use awk '!x++' inputfile.txt or sed '/pattern/d' or use uniq and sort... (7 Replies)
Discussion started by: redse171
7 Replies

8. UNIX for Dummies Questions & Answers

Grep from pattern file without removing duplicates?

I have been using grep to output whole lines using a pattern file with identifiers (fileA): fig|562.2322.peg.1 fig|562.2322.peg.3 fig|562.2322.peg.3 fig|562.2322.peg.3 fig|562.2322.peg.7 From fileB with corresponding identifiers in the second column: NODE_0 fig|562.2322.peg.1 peg ... (2 Replies)
Discussion started by: Mauve
2 Replies

9. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

10. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies
ZSHROADMAP(1)						      General Commands Manual						     ZSHROADMAP(1)

NAME
zshroadmap - informal introduction to the zsh manual The Zsh Manual, like the shell itself, is large and often complicated. This section of the manual provides some pointers to areas of the shell that are likely to be of particular interest to new users, and indicates where in the rest of the manual the documentation is to be found. WHEN THE SHELL STARTS
When it starts, the shell reads commands from various files. These can be created or edited to customize the shell. See the section Startup/Shutdown Files in zsh(1). If no personal initialization files exist for the current user, a function is run to help you change some of the most common settings. It won't appear if your administrator has disabled the zsh/newuser module. The function is designed to be self-explanatory. You can run it by hand with `autoload -Uz zsh-newuser-install; zsh-newuser-install -f'. See also the section User Configuration Functions in zshcon- trib(1). INTERACTIVE USE
Interaction with the shell uses the builtin Zsh Line Editor, ZLE. This is described in detail in zshzle(1). The first decision a user must make is whether to use the Emacs or Vi editing mode as the keys for editing are substantially different. Emacs editing mode is probably more natural for beginners and can be selected explicitly with the command bindkey -e. A history mechanism for retrieving previously typed lines (most simply with the Up or Down arrow keys) is available; note that, unlike other shells, zsh will not save these lines when the shell exits unless you set appropriate variables, and the number of history lines retained by default is quite small (30 lines). See the description of the shell variables (referred to in the documentation as parameters) HISTFILE, HISTSIZE and SAVEHIST in zshparam(1). The shell now supports the UTF-8 character set (and also others if supported by the operating system). This is (mostly) handled transpar- ently by the shell, but the degree of support in terminal emulators is variable. There is some discussion of this in the shell FAQ, http://zsh.dotsrc.org/FAQ/ . Note in particular that for combining characters to be handled the option COMBINING_CHARS needs to be set. Because the shell is now more sensitive to the definition of the character set, note that if you are upgrading from an older version of the shell you should ensure that the appropriate variable, either LANG (to affect all aspects of the shell's operation) or LC_CTYPE (to affect only the handling of character sets) is set to an appropriate value. This is true even if you are using a single-byte character set including extensions of ASCII such as ISO-8859-1 or ISO-8859-15. See the description of LC_CTYPE in zshparam(1). Completion Completion is a feature present in many shells. It allows the user to type only a part (usually the prefix) of a word and have the shell fill in the rest. The completion system in zsh is programmable. For example, the shell can be set to complete email addresses in argu- ments to the mail command from your ~/.abook/addressbook; usernames, hostnames, and even remote paths in arguments to scp, and so on. Any- thing that can be written in or glued together with zsh can be the source of what the line editor offers as possible completions. Zsh has two completion systems, an old, so called compctl completion (named after the builtin command that serves as its complete and only user interface), and a new one, referred to as compsys, organized as library of builtin and user-defined functions. The two systems differ in their interface for specifying the completion behavior. The new system is more customizable and is supplied with completions for many commonly used commands; it is therefore to be preferred. The completion system must be enabled explicitly when the shell starts. For more information see zshcompsys(1). Extending the line editor Apart from completion, the line editor is highly extensible by means of shell functions. Some useful functions are provided with the shell; they provide facilities such as: insert-composed-char composing characters not found on the keyboard match-words-by-style configuring what the line editor considers a word when moving or deleting by word history-beginning-search-backward-end, etc. alternative ways of searching the shell history replace-string, replace-pattern functions for replacing strings or patterns globally in the command line edit-command-line edit the command line with an external editor. See the section `ZLE Functions' in zshcontrib(1) for descriptions of these. OPTIONS
The shell has a large number of options for changing its behaviour. These cover all aspects of the shell; browsing the full documentation is the only good way to become acquainted with the many possibilities. See zshoptions(1). PATTERN MATCHING
The shell has a rich set of patterns which are available for file matching (described in the documentation as `filename generation' and also known for historical reasons as `globbing') and for use when programming. These are described in the section `Filename Generation' in zshexpn(1). Of particular interest are the following patterns that are not commonly supported by other systems of pattern matching: ** for matching over multiple directories ~, ^ the ability to exclude patterns from matching when the EXTENDED_GLOB option is set (...) glob qualifiers, included in parentheses at the end of the pattern, which select files by type (such as directories) or attribute (such as size). GENERAL COMMENTS ON SYNTAX
Although the syntax of zsh is in ways similar to the Korn shell, and therefore more remotely to the original UNIX shell, the Bourne shell, its default behaviour does not entirely correspond to those shells. General shell syntax is introduced in the section `Shell Grammar' in zshmisc(1). One commonly encountered difference is that variables substituted onto the command line are not split into words. See the description of the shell option SH_WORD_SPLIT in the section `Parameter Expansion' in zshexpn(1). In zsh, you can either explicitly request the splitting (e.g. ${=foo}) or use an array when you want a variable to expand to more than one word. See the section `Array Parameters' in zsh- param(1). PROGRAMMING
The most convenient way of adding enhancements to the shell is typically by writing a shell function and arranging for it to be autoloaded. Functions are described in the section `Functions' in zshmisc(1). Users changing from the C shell and its relatives should notice that aliases are less used in zsh as they don't perform argument substitution, only simple text replacement. A few general functions, other than those for the line editor described above, are provided with the shell and are described in zshcon- trib(1). Features include: promptinit a prompt theme system for changing prompts easily, see the section `Prompt Themes' zsh-mime-setup a MIME-handling system which dispatches commands according to the suffix of a file as done by graphical file managers zcalc a calculator zargs a version of xargs that makes the find command redundant zmv a command for renaming files by means of shell patterns. zsh 5.0.2 December 21, 2012 ZSHROADMAP(1)
All times are GMT -4. The time now is 06:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy