Sponsored Content
Full Discussion: Delete incomplete data
Top Forums UNIX for Dummies Questions & Answers Delete incomplete data Post 302930102 by RudiC on Tuesday 30th of December 2014 09:33:34 AM
Old 12-30-2014
Once the input file is sorted, you could run sth like
Code:
awk     'NR==1          {P=$1}
         P != $1        {if (N && D) print O; O=DL=""; N=0; D=0; P=$1}
                        {O=O DL $0; DL="\n"}
         $2 ~ /name$/   {N=1}
         $2 ~ /^Deta/   {D=1}
         END            {if (N && D) print O}
        ' FS="," file
Person1,Firstname,x1
Person1,Lastname,x2
Person1,Details,x3
Person1,Details,x4
Person2,Firstname,x5
Person2,Details,x6
Person2,Details,x7
Person2,Details,x8
Person3,Details,x9
Person3,Lastname,x10
Person3,Details,x11
Person3,Details,x12
Person6,Firstname,x5
Person6,Details,x6

This will work happily through your file without requiring too much memory; still it may take its time.
This User Gave Thanks to RudiC For This Post:
 

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete blocks with no data..

Hi, I tried this but could not get it... here is what I need I have an xml where I get all the data in blocks but some times I get empty blocks with no data...shown below..I need to delete only those blocks with no data, I tried couple of ways but could not do it..any help is appreciated...... (1 Reply)
Discussion started by: mgirinath
1 Replies

2. Programming

edit, and delete data from you file

Hi, I'm new baby in linux and i need your help to make a script. i have a file like this: nameuser Password="password-user" Data-profile=tipe-profile, IP-Address=ip-data, country=country_user, old-user=old_user Scripts I need that allow search, edit, and delete data from you file. ... (1 Reply)
Discussion started by: dorek
1 Replies

3. Shell Programming and Scripting

Delete unmatched data

Hi, I try to write script to compare 2 data file (list of numbers) which after that I want to delete unmatched numbers and create new file for matched numbers. Can anybody to help me? (5 Replies)
Discussion started by: nazri76
5 Replies

4. Shell Programming and Scripting

Delete all data before first space occurence

Hi Guyz, Can anyone help me in the following:- I want to delete all that is there in my file that occures before the first space comes. Eg. My file InputFile conatins the following: 123 12345678 87654 Hello 09867 09876654 34567 Happy I want the data occuring before the occurence... (3 Replies)
Discussion started by: DTechBuddy
3 Replies

5. Shell Programming and Scripting

Reference data check for delete

Dear All, I have a master file - Master.txt 100|ABC 200|CED 500|XYZ 800|POL I have a reference file - Ref.txt 200 800 What is desired.. Check for all those records in reference file matching with those within master file and then delete those records from Master file So, at end,... (1 Reply)
Discussion started by: sureshg_sampat
1 Replies

6. Shell Programming and Scripting

Delete duplicate data and pertain the latest month data.

Hi I have a file with following records It contains three months of data, some data is duplicated,i need to access the latest data from the duplicate ones. for e.g; i have foll data "200","0","","11722","-63","","","","11722","JUL","09" "200","0","","11722","-63","","","","11722","JUL","09"... (10 Replies)
Discussion started by: vee_789
10 Replies

7. Shell Programming and Scripting

Delete data between [] with sed

i'm cat /var/log/message Jan 10 14:48:45 LOG SKYPE-OUT IN=eth1 OUT=eth2 SRC=192.168.1.65 DST=203.157.168.5 PROTO=TCP SPT=1284 DPT=3306 Jan 10 14:48:45 LOG HTTPS IN=eth0 OUT=eth1 SRC=207.46.15.251 DST=192.168.1.47 PROTO=TCP SPT=443 DPT=2069 Jan 10 14:48:45 LOG HTTPS IN=eth0 OUT=eth1... (2 Replies)
Discussion started by: slackman
2 Replies

8. UNIX for Dummies Questions & Answers

Mapping a data in a file and delete line in source file if data does not exist.

Hi Guys, Please help me with my problem here: I have a source file: 1212 23232 343434 ASAS1 4 3212 23232 343434 ASAS2 4 3234 23232 343434 QWQW1 4 1134 23232 343434 QWQW2 4 3212 23232 343434 QWQW3 4 and a mapping... (4 Replies)
Discussion started by: kokoro
4 Replies
CANONICAL(5)							File Formats Manual						      CANONICAL(5)

NAME
canonical - Postfix canonical table format SYNOPSIS
postmap /etc/postfix/canonical postmap -q "string" /etc/postfix/canonical postmap -q - /etc/postfix/canonical <inputfile DESCRIPTION
The optional canonical(5) table specifies an address mapping for local and non-local addresses. The mapping is used by the cleanup(8) dae- mon, before mail is stored into the queue. The address mapping is recursive. Normally, the canonical(5) table is specified as a text file that serves as input to the postmap(1) command. The result, an indexed file in dbm or db format, is used for fast searching by the mail system. Execute the command "postmap /etc/postfix/canonical" to rebuild an indexed file after changing the corresponding text file. When the table is provided via other means such as NIS, LDAP or SQL, the same lookups are done as for ordinary indexed files. Alternatively, the table can be provided as a regular-expression map where patterns are given as regular expressions, or lookups can be directed to TCP-based server. In those cases, the lookups are done in a slightly different way as described below under "REGULAR EXPRESSION TABLES" or "TCP-BASED TABLES". By default the canonical(5) mapping affects both message header addresses (i.e. addresses that appear inside messages) and message envelope addresses (for example, the addresses that are used in SMTP protocol commands). This is controlled with the canonical_classes parameter. NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP clients only if the client matches the local_header_re- write_clients parameter, or if the remote_header_rewrite_domain configuration parameter specifies a non-empty value. To get the behavior before Postfix 2.2, specify "local_header_rewrite_clients = static:all". Typically, one would use the canonical(5) table to replace login names by Firstname.Lastname, or to clean up addresses produced by legacy mail systems. The canonical(5) mapping is not to be confused with virtual alias support or with local aliasing. To change the destination but not the headers, use the virtual(5) or aliases(5) map instead. CASE FOLDING
The search string is folded to lowercase before database lookup. As of Postfix 2.3, the search string is not case folded with database types such as regexp: or pcre: whose lookup fields can match both upper and lower case. TABLE FORMAT
The input format for the postmap(1) command is as follows: pattern result When pattern matches a mail address, replace it by the corresponding result. blank lines and comments Empty lines and whitespace-only lines are ignored, as are lines whose first non-whitespace character is a `#'. multi-line text A logical line starts with non-whitespace text. A line that starts with whitespace continues a logical line. TABLE SEARCH ORDER
With lookups from indexed files such as DB or DBM, or from networked tables such as NIS, LDAP or SQL, patterns are tried in the order as listed below: user@domain address Replace user@domain by address. This form has the highest precedence. This is useful to clean up addresses produced by legacy mail systems. It can also be used to produce Firstname.Lastname style addresses, but see below for a simpler solution. user address Replace user@site by address when site is equal to $myorigin, when site is listed in $mydestination, or when it is listed in $inet_interfaces or $proxy_interfaces. This form is useful for replacing login names by Firstname.Lastname. @domain address Replace other addresses in domain by address. This form has the lowest precedence. Note: @domain is a wild-card. When this form is applied to recipient addresses, the Postfix SMTP server accepts mail for any recipi- ent in domain, regardless of whether that recipient exists. This may turn your mail system into a backscatter source: Postfix first accepts mail for non-existent recipients and then tries to return that mail as "undeliverable" to the often forged sender address. RESULT ADDRESS REWRITING
The lookup result is subject to address rewriting: o When the result has the form @otherdomain, the result becomes the same user in otherdomain. o When "append_at_myorigin=yes", append "@$myorigin" to addresses without "@domain". o When "append_dot_mydomain=yes", append ".$mydomain" to addresses without ".domain". ADDRESS EXTENSION
When a mail address localpart contains the optional recipient delimiter (e.g., user+foo@domain), the lookup order becomes: user+foo@domain, user@domain, user+foo, user, and @domain. The propagate_unmatched_extensions parameter controls whether an unmatched address extension (+foo) is propagated to the result of table lookup. REGULAR EXPRESSION TABLES
This section describes how the table lookups change when the table is given in the form of regular expressions. For a description of regu- lar expression lookup table syntax, see regexp_table(5) or pcre_table(5). Each pattern is a regular expression that is applied to the entire address being looked up. Thus, user@domain mail addresses are not broken up into their user and @domain constituent parts, nor is user+foo broken up into user and foo. Patterns are applied in the order as specified in the table, until a pattern is found that matches the search string. Results are the same as with indexed file lookups, with the additional feature that parenthesized substrings from the pattern can be inter- polated as $1, $2 and so on. TCP-BASED TABLES This section describes how the table lookups change when lookups are directed to a TCP-based server. For a description of the TCP client/server lookup protocol, see tcp_table(5). This feature is not available up to and including Postfix version 2.4. Each lookup operation uses the entire address once. Thus, user@domain mail addresses are not broken up into their user and @domain con- stituent parts, nor is user+foo broken up into user and foo. Results are the same as with indexed file lookups. BUGS
The table format does not understand quoting conventions. CONFIGURATION PARAMETERS
The following main.cf parameters are especially relevant. The text below provides only a parameter summary. See postconf(5) for more details including examples. canonical_classes What addresses are subject to canonical address mapping. canonical_maps List of canonical mapping tables. recipient_canonical_maps Address mapping lookup table for envelope and header recipient addresses. sender_canonical_maps Address mapping lookup table for envelope and header sender addresses. propagate_unmatched_extensions A list of address rewriting or forwarding mechanisms that propagate an address extension from the original address to the result. Specify zero or more of canonical, virtual, alias, forward, include, or generic. Other parameters of interest: inet_interfaces The network interface addresses that this system receives mail on. You need to stop and start Postfix when this parameter changes. local_header_rewrite_clients Rewrite message header addresses in mail from these clients and update incomplete addresses with the domain name in $myorigin or $mydomain; either don't rewrite message headers from other clients at all, or rewrite message headers and update incomplete addresses with the domain specified in the remote_header_rewrite_domain parameter. proxy_interfaces Other interfaces that this machine receives mail on by way of a proxy agent or network address translator. masquerade_classes List of address classes subject to masquerading: zero or more of envelope_sender, envelope_recipient, header_sender, header_recipi- ent. masquerade_domains List of domains that hide their subdomain structure. masquerade_exceptions List of user names that are not subject to address masquerading. mydestination List of domains that this mail system considers local. myorigin The domain that is appended to locally-posted mail. owner_request_special Give special treatment to owner-xxx and xxx-request addresses. remote_header_rewrite_domain Don't rewrite message headers from remote clients at all when this parameter is empty; otherwise, rewrite message headers and append the specified domain name to incomplete addresses. SEE ALSO
cleanup(8), canonicalize and enqueue mail postmap(1), Postfix lookup table manager postconf(5), configuration parameters virtual(5), virtual aliasing README FILES
Use "postconf readme_directory" or "postconf html_directory" to locate this information. DATABASE_README, Postfix lookup table overview ADDRESS_REWRITING_README, address rewriting guide LICENSE
The Secure Mailer license must be distributed with this software. AUTHOR(S) Wietse Venema IBM T.J. Watson Research P.O. Box 704 Yorktown Heights, NY 10598, USA CANONICAL(5)
All times are GMT -4. The time now is 02:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy