Sponsored Content
Full Discussion: Avoid Duplicates in a file
Top Forums UNIX for Dummies Questions & Answers Avoid Duplicates in a file Post 302178074 by lazytech on Monday 24th of March 2008 02:33:06 PM
Old 03-24-2008
My suggestion would be to write a Perl script incorporating Regular Expression to solve your issue. It may be easier to pull out your unique values that way.
 

10 More Discussions You Might Find Interesting

1. Ubuntu

Avoid creating temporary files on editing a file in Ubuntu

Hi, My ubuntu flavor always create temporary files having filename followed by ~ on editing. For eg: if I am editing a file called "sip.c", automatically a temporary (bkup) file is getting created with the name "sip.c~". How to avoid this file creation? (7 Replies)
Discussion started by: royalibrahim
7 Replies

2. Shell Programming and Scripting

How to avoid a temp file

Hi all. I want to check the free space on a given FS and process the output. Right now, I'm using a temp file to avoid using df twice. This is what I'm doing #!/usr/bin/ksh ... df -k $FS_NAME > $TMP_FILE 2>&1 if ]; then RESULT="CRITICAL - $(cat $TMP_FILE)" else cat $TMP_FILE | ...... (3 Replies)
Discussion started by: fox1212
3 Replies

3. HP-UX

Parameter to avoid file being deleted by SAM

Good afternoon. I am a newbie. We just had a potentially big problem (negated to having good backups). Basically, there is an option in SAM, to delete all the data from the system that a user ever created. Lo and behold, silly me, I choose that option, and all sorts of needed files... (5 Replies)
Discussion started by: instant000
5 Replies

4. Shell Programming and Scripting

Avoid file creation in a script...achive same result

Guys following lines help me in getting numbers from PID column ,to be thrown into first column of a CSV file. COLUMNS=2047 /usr/bin/ps -eo pid,ppid,uid,user,args | grep -v "PID" > /tmp/masterPID.txt cat /tmp/masterPID.txt|while read line do PID=`echo $line|awk '{print $1}'` echo "$PID"... (4 Replies)
Discussion started by: ak835
4 Replies

5. Shell Programming and Scripting

avoid open file to check field.

Hi Everyone, # cat a.txt 94,aqqc,62345907, 5,aeec,77, # cat 1.pl #!/usr/bin/perl use strict; use warnings; use Date::Manip; open(my $FA, "/root/a.txt") or die "$!"; while(<$FA>) { chomp; my @tmp=split(/\,/, $_); if (index($tmp, "qq") ne -1) { ... (4 Replies)
Discussion started by: jimmy_y
4 Replies

6. Shell Programming and Scripting

How to avoid Newline character in generated text file?

Hi All, Just need small help in resolving the special new line character in generated output file. In one of my shell script I am using following lines to get the spool file (i.e. sfile.txt) and AAA_XXXX_p111_n222.txt AAA_YYYY_p111_n222.txt Here assuming v_pnum="p111" v_nid="n222" ... (1 Reply)
Discussion started by: shekharjchandra
1 Replies

7. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

8. Programming

[c]Why first file is creating after the second. How to avoid

Hi, My Code is as below: nbECRITS = fwrite(strstr(data->buffer, ";") + 1, sizeof(char), (data->buffsize) - LEN_NOM_FIC, fic_sortie); fclose(fic_sortie); sprintf(PATH_BALISE, "%s.balise", PATH); fic_balise_data = fopen(PATH_BALISE, "a+"); if (fic_balise_data == NULL) {... (1 Reply)
Discussion started by: ezee
1 Replies

9. Shell Programming and Scripting

Avoid overwriting backup file when multiple entries need to replace in one file input from another

Hello, I have been working on script in which search and replace the multiple pattern. 1. update_params.sh read the multiple pattern from input file ParamMapping.txt(old_entry|New_entry) and passing this values one by one to change_text.sh 2. change_text.sh read... (0 Replies)
Discussion started by: ketanraut
0 Replies

10. Shell Programming and Scripting

Write only changes to file - avoid duplicates

I want to create a file, to save a list of fail2ban blocked ip addresses. So I thought I'd create a loop that will check with fail2ban every minute, and write the ip addresses to a file. while true; do echo $(fail2ban-client status asterisk-iptables | grep 'IP list' | sed 's/.*://g' | sed -e... (4 Replies)
Discussion started by: aristosv
4 Replies
AMSTAR(8)						  System Administration Commands						 AMSTAR(8)

NAME
amstar - Amanda Application to interface with star DESCRIPTION
Amstar is an Amanda Application API script. It should not be run by users directly. It uses star to backup and restore data. The diskdevice in the disklist (DLE) must be the directory to backup. INCLUDE - EXCLUDE amstar doesn't support include. Both 'exclude list' and 'exclude file' are supported. There is a maximum of 100 patterns (limit of star). Moving directories into and out of directories on the exclude list causes incremental backups to not contain the files in the moved directories until the next full backup unless the moved files are modified after the move. PROPERTIES
This section lists the properties that control amstar's functionality. See amanda-applications(7) for information on application properties and how they are configured. ACL If "YES" (the default), amstar will store files acl by passing the -acl argument to star. If "NO", then the -acl option is not given to star, and it will not try to backup the acl. COMMAND-OPTIONS If set, theses options are passed asis to star. Each option must be a different value of the property. Some option can break how amanda do backup, use it with care. Use: property "COMMAND-OPTIONS" "--foo" "bar" Do not use: property "COMMAND-OPTIONS" "--foo bar" DIRECTORY If set, gnutar will backup from that directory instead of the diskdevice set by the DLE. On restore, the data is restore in that directpory instead of the current working directory. ONE-FILE-SYSTEM The default is "YES". This property is deprecated and the default value can't be changed. star requires it when it works in dump mode. This corresponds to the -xdev option of star. SPARSE If "YES" (the default), star will store sparse files efficiently. If "NO", then the -sparse option is not given to star, and it will not try to detect sparse files. STAR-DLE-TARDUMP If "YES", amstar will use a different tardump file for each DLE. The default is "NO". This property is needed with older versions of star doing many dumps in parallel, because of a race condition in updating the tardump file. STAR-PATH The path to the star binary. The default is set when Amanda is built by the --with-star configure option. STAR-TARDUMP The directory where star stores the database it uses to generate incremental dumps. The default is /etc/tardumps. NORMAL List all regex (POSIX Extended Regular Expression syntax) that are normal output from star. These output are in the "FAILED DUMP DETAILS" section of the email report if the dump result is STRANGE or FAILED. Default values: "^could not open conf file" "^Type of this level " "^Date of this level " "^Date of last level " "^Dump record level " "^Throughput" "^.*is sparse$" "^star: dumped [0-9][0-9]* (tar )?files" "^.*The following problems occurred during .* processing.*$" "^.*Processed all possible files, despite earlier errors.*$" "^.*not written due to problems during backup.*$" To treat one of these default patterns differently, specify it explicitly in a different property. IGNORE List all regex (POSIX Extended Regular Expression syntax) that amanda ignore. These output are never in the email report. STRANGE List all regex (POSIX Extended Regular Expression syntax) that are strange output from star. All star output that doesn't match a normal or ignore regex are strange by default. The result of the dump is STRANGE if star produce a strange output. These output are in the "FAILED DUMP DETAILS" section of the email report. "^Perform a level 0 dump first.*$" To treat one of these default patterns differently, specify it explicitly in a different property. SEE ALSO
amanda(8), amanda.conf(5), amanda-applications(7) The Amanda Wiki: : http://wiki.zmanda.com/ AUTHORS
Jean-Louis Martineau <martineau@zmanda.com> Zmanda, Inc. (http://www.zmanda.com) Dustin J. Mitchell <dustin@zmanda.com> Zmanda, Inc. (http://www.zmanda.com) Amanda 3.3.1 02/21/2012 AMSTAR(8)
All times are GMT -4. The time now is 08:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy