Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Deleting duplicated chunks in a file using awk/sed
Top Forums Shell Programming and Scripting Deleting duplicated chunks in a file using awk/sed Post 302974132 by Don Cragun on Thursday 26th of May 2016 12:12:04 AM
Old 05-26-2016
Instead of trying to get multiple websites to act as your unpaid programming staff, why don't you show us how you have tried to solved this problem on your own? If you can show us what you have tried, maybe we can help you fix it.

We have helped you with 8 other awk scripts in the last six months. Can't you use the examples provided by those scripts to get a good start on what you need here?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using sed to get rid of duplicated columns...

I can not figure out this one, so I turn to unix.com for help, I have a file, in which there are some lines containing continuously duplicate columns, like the following adb abc abc asd adfj 123 123 123 345 234 444 444 444 444 444 23 and the output I want is adb abc asd adfj 123 345... (5 Replies)
Discussion started by: fedora
5 Replies

2. Shell Programming and Scripting

Deleting Doubled Block with sed or awk

hi there, i have a text file like that one: I like to delete the second block with the Start and End Line! Does anyone have a idea? Thanks for your help, Roland (4 Replies)
Discussion started by: rolandh
4 Replies

3. Shell Programming and Scripting

Deleting a line from a file with sed and awk?

cat file.txt fvnuiehuewf ruevhxncvkjrh zxjvurhfuwe jkhvBEGINvfnvf ijrgioe Trying to delete a line that has the pattern "BEGIN" cat sedtest filename=file.txt pattern=BEGIN sed "/^$pattern/d" "$filename" (9 Replies)
Discussion started by: cola
9 Replies

4. Shell Programming and Scripting

Deleting the first column with sed,awk or perl

336 brtr 256 hello Output: brtr hello How can i do this with sed,awk or perl? (5 Replies)
Discussion started by: cola
5 Replies

5. Shell Programming and Scripting

Deleting characters with sed,perl,awk

Input: :: gstreamer :: xine-lib :: xine-lib-extras Output should be: gstreamer xine-lib xine-lib-extras How can it be done with sed or perl? (12 Replies)
Discussion started by: cola
12 Replies

6. Shell Programming and Scripting

Can I use a shell script for deleting chunks from a watch folder?

Hello I have a unique problem of needing to delete large files slowly off of an XSan. I was wondering if there is a script I could use to delete 100gb chunks of files and folders that get placed in to a watch folder, slowly so as not to disrupt the other users. I would like to use Automator in... (0 Replies)
Discussion started by: ajsoto
0 Replies

7. Shell Programming and Scripting

deleting lines between patterns using sed or awk

hi, Here is excerpt from my xml file <!-- The custom module to do the authentication for LDAP --> </login-module> <login-module code="com.nlayers.seneca.security.LdapLogin" flag="sufficient"> <module-option... (1 Reply)
Discussion started by: sunrexstar
1 Replies

8. UNIX for Dummies Questions & Answers

Awk: Print out overlapping chunks of file - rows 0-20,10-30,20-40 etc.

First time poster, but the forum has saved my bacon more times than... Lots. Anyway, I have a text file, and wanted to use Awk (or any other sensible program) to print out overlapping sections, or arbitrary length. To describe by example, for file 1 2 3 4 5 etc... I want the out put... (3 Replies)
Discussion started by: matfald
3 Replies

9. Shell Programming and Scripting

awk for splitting file in constant chunks

Hi gurus, I wanted to split main file in 20 files with 2500 lines in each file. My main file conatins total 2500*20 lines. Following awk I made, but it is breaking with error. awk '{ for (i = 1; i <= 20; i++) { starts=2500*$i-1; ends=2500*$i; NR>=starts && NR<=ends {f=My$i".txt"; print >> f;... (10 Replies)
Discussion started by: mukesh.lalwani
10 Replies

10. Shell Programming and Scripting

Deleting lines containing duplicated strings

Dear all, I always appreciate your help. I would like to delete lines containing duplicated strings in the second column. test.txt 658 invert_d2e_q_reg_0_/Qalu_ecl_zlow_e 0.825692 659 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692 660 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692... (1 Reply)
Discussion started by: jypark22
1 Replies

LEARN ABOUT MINIX

dh_usrlocal

DH_USRLOCAL(1)							     Debhelper							    DH_USRLOCAL(1)

NAME

       dh_usrlocal - migrate usr/local directories to maintainer scripts

SYNOPSIS

       dh_usrlocal [debhelperoptions] [-n]

DESCRIPTION

       dh_usrlocal is a debhelper program that can be used for building packages that will provide a subdirectory in /usr/local when installed.

       It finds subdirectories of usr/local in the package build directory, and removes them, replacing them with maintainer script snippets
       (unless -n is used) to create the directories at install time, and remove them when the package is removed, in a manner compliant with
       Debian policy. These snippets are inserted into the maintainer scripts by dh_installdeb. See dh_installdeb(1) for an explanation of
       debhelper maintainer script snippets.

       When the Rules-Requires-Root field is not (effectively) binary-targets, the directories in /usr/local will have ownership root:staff and
       the mode will be 02775.	These values have been chosen to comply with the recommendations of the Debian policy for directories in
       /usr/local.

       When Rules-Requires-Root has an effective value of binary-targets, the owners, groups and permissions will be preserved with one exception.
       If the directory is owned by root:root, then ownership will be reset to root:staff and mode will be reset to 02775. This is useful, since
       that is the group and mode policy recommends for directories in /usr/local.

OPTIONS

       -n, --no-scripts
	   Do not modify postinst/prerm scripts.

NOTES

       Note that this command is not idempotent. dh_prep(1) should be called between invocations of this command. Otherwise, it may cause multiple
       instances of the same text to be added to maintainer scripts.

CONFORMS TO

       Debian policy, version 2.2

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Andrew Stribblehill <ads@debian.org>

11.1.6ubuntu2							    2018-05-10							    DH_USRLOCAL(1)
All times are GMT -4. The time now is 11:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy