Sponsored Content
Top Forums Shell Programming and Scripting Delete duplicates via script? Post 302463574 by frans on Monday 18th of October 2010 03:45:09 AM
Old 10-18-2010
Simple
shell code:
  1. while read file
  2. do rm -f FolderB/$file
  3. done < <(ls FolderA)
This User Gave Thanks to frans For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

An interactive way to delete duplicates

1)I am trying to write a script that works interactively lists duplicated records on certain field/column and asks user to delete one or more. And finally it deletes all the records the used has asked for. I have an idea to store those line numbers in an array, not sure how to do this in... (3 Replies)
Discussion started by: chvs2000
3 Replies

2. Shell Programming and Scripting

How can i delete the duplicates based on one column of a line

I have my data something like this (08/03/2009 22:57:42.414)(:) king aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb (08/03/2009 22:57:42.416)(:) John cccccccccccc cccccvssssssssss baaaaa (08/03/2009 22:57:42.417)(:) Michael ddddddd tststststtststts (08/03/2009 22:57:42.425)(:) Ravi... (11 Replies)
Discussion started by: rdhanek
11 Replies

3. Shell Programming and Scripting

how can I delete duplicates in the log?

I have a log file and I am trying to run a script against it to search for key issues such as invalid users, errors etc. In one part, I grep for session closed and get a lot of the same thing,, ie. root username etc. I want to remove the multiple root and just have it do a count, like wc -l ... (5 Replies)
Discussion started by: taekwondo
5 Replies

4. Shell Programming and Scripting

Delete Duplicates on the basis of two column values.

Hi All, i need ti delete two duplicate processss which are running on the same device type (column 1) and port ID (column 2). here is the sample data p1sc1m1 15517 11325 0 01:00:24 ? 0:00 scagntclsx25octtcp 2967 in3v mvmp01 0 8000 N S 969 750@751@752@ p1sc1m1 15519 11325 0 01:00:24 ? ... (5 Replies)
Discussion started by: neeraj617
5 Replies

5. Shell Programming and Scripting

Fastest way to delete duplicates from a large filelist.....

OK I have two filelists...... The first is formatted like this.... /path/to/the/actual/file/location/filename.jpg and has up to a million records The second list shows filename.jpg where there is more then on instance. and has maybe up to 65,000 records I want to copy files... (4 Replies)
Discussion started by: Bashingaway
4 Replies

6. Shell Programming and Scripting

delete from line and remove duplicates

My Input.....file1 ABCDE4435 Connected to 107.71.136.122 (SubNetwork=ONRM_RootMo_R SubNetwork=XYVLTN29CRBR99 MeContext=ABCDE4435 ManagedElement=1) ABCDE4478 Connected to 166.208.30.57 (SubNetwork=ONRM_RootMo_R SubNetwork=KLFMTN29CR0R04 MeContext=ABCDE4478 ManagedElement=1) ABCDE4478... (5 Replies)
Discussion started by: pareshkp
5 Replies

7. Shell Programming and Scripting

Delete duplicates in CA bundle

I do have a big CA bundle certificate file and each time if i get request to add new certificate to the existing bundle i need to make sure it is not present already. How i can validate the duplicates. The alignment of the certificate within the bundle seems to be different. Example: Cert 1... (7 Replies)
Discussion started by: diva_thilak
7 Replies

8. Shell Programming and Scripting

Delete only if duplicates found in each record

Hi, i have another problem. I have been trying to solve it by myself but failed. inputfile ;; ID T08578 NAME T08578 SBASE 30696 EBASE 32083 TYPE P func just test func chronology func cholesterol func null INT 30765-37333 INT 37154-37318 Link 5546 Link 8142 (4 Replies)
Discussion started by: redse171
4 Replies

9. Shell Programming and Scripting

Script to compare partial filenames in two folders and delete duplicates

Background: I use a TV tuner card to capture OTA video files (.mpeg) and then my Plex Media Server automatically optimizes the files (transcodes for better playback) and places them in a new directory. I have another Plex Library pointing to the new location for the optimized .mp4 files. This... (2 Replies)
Discussion started by: shaky
2 Replies

10. Shell Programming and Scripting

To Delete the duplicates using Part of File Name

I am using the below script to delete duplicate files but it is not working for directories with more than 10k files "Argument is too long" is getting for ls -t. Tried to replace ls -t with find . -type f \( -iname "*.xml" \) -printf '%T@ %p\n' | sort -rg | sed -r 's/* //' | awk... (8 Replies)
Discussion started by: gold2k8
8 Replies
AutoSplit(3pm)						 Perl Programmers Reference Guide					    AutoSplit(3pm)

NAME
AutoSplit - split a package for autoloading SYNOPSIS
autosplit($file, $dir, $keep, $check, $modtime); autosplit_lib_modules(@modules); DESCRIPTION
This function will split up your program into files that the AutoLoader module can handle. It is used by both the standard perl libraries and by the MakeMaker utility, to automatically configure libraries for autoloading. The "autosplit" interface splits the specified file into a hierarchy rooted at the directory $dir. It creates directories as needed to reflect class hierarchy, and creates the file autosplit.ix. This file acts as both forward declaration of all package routines, and as timestamp for the last update of the hierarchy. The remaining three arguments to "autosplit" govern other options to the autosplitter. $keep If the third argument, $keep, is false, then any pre-existing "*.al" files in the autoload directory are removed if they are no longer part of the module (obsoleted functions). $keep defaults to 0. $check The fourth argument, $check, instructs "autosplit" to check the module currently being split to ensure that it includes a "use" specifi- cation for the AutoLoader module, and skips the module if AutoLoader is not detected. $check defaults to 1. $modtime Lastly, the $modtime argument specifies that "autosplit" is to check the modification time of the module against that of the "autosplit.ix" file, and only split the module if it is newer. $modtime defaults to 1. Typical use of AutoSplit in the perl MakeMaker utility is via the command-line with: perl -e 'use AutoSplit; autosplit($ARGV[0], $ARGV[1], 0, 1, 1)' Defined as a Make macro, it is invoked with file and directory arguments; "autosplit" will split the specified file into the specified directory and delete obsolete ".al" files, after checking first that the module does use the AutoLoader, and ensuring that the module is not already currently split in its current form (the modtime test). The "autosplit_lib_modules" form is used in the building of perl. It takes as input a list of files (modules) that are assumed to reside in a directory lib relative to the current directory. Each file is sent to the autosplitter one at a time, to be split into the directory lib/auto. In both usages of the autosplitter, only subroutines defined following the perl __END__ token are split out into separate files. Some rou- tines may be placed prior to this marker to force their immediate loading and parsing. Multiple packages As of version 1.01 of the AutoSplit module it is possible to have multiple packages within a single file. Both of the following cases are supported: package NAME; __END__ sub AAA { ... } package NAME::option1; sub BBB { ... } package NAME::option2; sub BBB { ... } package NAME; __END__ sub AAA { ... } sub NAME::option1::BBB { ... } sub NAME::option2::BBB { ... } DIAGNOSTICS
"AutoSplit" will inform the user if it is necessary to create the top-level directory specified in the invocation. It is preferred that the script or installation process that invokes "AutoSplit" have created the full directory path ahead of time. This warning may indicate that the module is being split into an incorrect path. "AutoSplit" will warn the user of all subroutines whose name causes potential file naming conflicts on machines with drastically limited (8 characters or less) file name length. Since the subroutine name is used as the file name, these warnings can aid in portability to such systems. Warnings are issued and the file skipped if "AutoSplit" cannot locate either the __END__ marker or a "package Name;"-style specification. "AutoSplit" will also emit general diagnostics for inability to create directories or files. perl v5.8.0 2002-06-01 AutoSplit(3pm)
All times are GMT -4. The time now is 01:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy