Sponsored Content
Top Forums Shell Programming and Scripting remove chunks of text from file Post 302339115 by staze on Wednesday 29th of July 2009 07:43:16 PM
Old 07-29-2009
remove chunks of text from file

All,

So, I have an ldif file that contains about 6500 users worth of data. Some users have a block of text I'd like to remove, while some don't.

Example (block of text in question is the block starting with "authAuthority: ;Kerberosv5"):

User with text block:

Code:
# username, users, example.com
dn: uid=username,cn=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
objectClass: apple-user
objectClass: extensibleObject
objectClass: organizationalPerson
objectClass: top
objectClass: person
apple-generateduid: 53CA02D7-B116-4461-B220-E3FC0B15964A
apple-mcxflags:: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPCFET0NUW
 VBFIHBsaXN0IFBVQkxJQyAiLS8vQXBwbGUgQ29tcHV0ZXIvL0RURCBQTElTVCAxLjAvL0VOIiAiaH
 R0cDovL3d3dy5hcHBsZS5jb20vRFREcy9Qcm9wZXJ0eUxpc3QtMS4wLmR0ZCI+CjxwbGlzdCB2ZXJ
 zaW9uPSIxLjAiPgo8ZGljdD4KCTxrZXk+c2ltdWx0YW5lb3VzX2xvZ2luX2VuYWJsZWQ8L2tleT4K
 CTx0cnVlLz4KPC9kaWN0Pgo8L3BsaXN0Pgo=
loginShell: /bin/bash
uidNumber: 20192
authAuthority: ;ApplePasswordServer;0x470bb9eb325f31c3000040ee00002257,1024 35
  1423486873699801821345071757674738484067280188359389504392445041998105914670
 84867869429532763785664902803450035110236201552277202539905523086333992178101
 54867353409493808376385021788117196022631658234104675864712197939496802664455
 87225827331332464303631278838001920713257416459820742251056515142078124405645
 79 root@example.com:123.456.789.111
authAuthority: ;Kerberosv5;0x470bb9eb325f31c3000040ee00002257;username@EXAMPLE.C
 OM;EXAMPLE.COM;1024 35 142348687369980182134507175767473848406728
 01883593895043924450419981059146708486786942953276378566490280345003511023620
 15522772025399055230863339921781015486735340949380837638502178811719602263165
 82341046758647121979394968026644558722582733133246430363127883800192071325741
 645982074225105651514207812440564579 root@example.com:123.456.789.111
userPassword:: KioqKioqKio=
uid: username
cn: Firstname Lastname
gidNumber: 1029
givenName: Firstname
sn: Lastname
apple-user-homeurl:: PGhvbWVfZGlyPjx1cmw+YWZwOi8vamRhdGExLnVvcmVnb24uZWR1L1VzZ
 XJzPC91cmw+PHBhdGg+c3R1cmNvPC9wYXRoPjwvaG9tZV9kaXI+
homeDirectory: /Network/Servers/example.com/Users/username
apple-user-homequota: 4294967296
mail: username@example.com

Now, one problem is, ldapsearch/ldapdump break up attributes at 76 characters. So, the block in question should be one line.

So I'm curious if there's an easy way to either A. remove the line breaks for the blocks of text (any line that starts with a " " should have the space removed, and should be on the line above. Though, one line starts with " " and only should have one " " removed then get put back with the previous line, or B. just to nuke the whole block of text that starts with "authAuthority: ;Kerberosv5" and ends with "example.com:123.456.789.111".

Anyone have any ideas? (btw, I realize that the line breaks aren't at exactly 76 anymore, since I had to sterilize the text for any personal info).

Last edited by staze; 07-29-2009 at 11:46 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove specified text from file

I am trying to write a script that kills old sessions, I've posted here over the past few days and the script is just about perfect except I want to be given the option to exclude specified PIDs from being killed. this is the entire script: if then rm /tmp/idlepids fi if then rm... (2 Replies)
Discussion started by: raidzero
2 Replies

2. Shell Programming and Scripting

Remove filename is text file

Hello, I got files full path in a text file like that /main/k/kdelibs/kdelibs4c2a_3.5.10.dfsg.1-2ubuntu7_i386.deb /main/k/kdelibs-experimental/libknotificationitem-dev_4.3.2-0ubuntu1_i386.deb /main/k/kdemultimedia/dragonplayer_4.3.2-0ubuntu1_i386.deb... (13 Replies)
Discussion started by: davidkhan
13 Replies

3. Shell Programming and Scripting

Split file into chunks of low & high byte

Hi guys, i have a question about spliting a binary file into 2 chunks. First chunk with all high bytes and the second one with all low bytes. What unix tools can i use? And how can this be performed? I looked in manpages of split and dd but this does not help. Thanks (2 Replies)
Discussion started by: basta
2 Replies

4. Shell Programming and Scripting

Parsing chunks of text and finding data

Hi, I need a script that parses and greps data out of a textfile. I have a text file that has this structure: File1 host1.localdomain text random text Found errors this text is random (41123) --- random random at.5165 ---- random random at.5165 ---- random random at.5165 ----... (2 Replies)
Discussion started by: erick_tuk
2 Replies

5. UNIX for Dummies Questions & Answers

Awk: Print out overlapping chunks of file - rows 0-20,10-30,20-40 etc.

First time poster, but the forum has saved my bacon more times than... Lots. Anyway, I have a text file, and wanted to use Awk (or any other sensible program) to print out overlapping sections, or arbitrary length. To describe by example, for file 1 2 3 4 5 etc... I want the out put... (3 Replies)
Discussion started by: matfald
3 Replies

6. Shell Programming and Scripting

awk for splitting file in constant chunks

Hi gurus, I wanted to split main file in 20 files with 2500 lines in each file. My main file conatins total 2500*20 lines. Following awk I made, but it is breaking with error. awk '{ for (i = 1; i <= 20; i++) { starts=2500*$i-1; ends=2500*$i; NR>=starts && NR<=ends {f=My$i".txt"; print >> f;... (10 Replies)
Discussion started by: mukesh.lalwani
10 Replies

7. Shell Programming and Scripting

Reverse sort on delimited chunks within a file

Hello, I have a large file in which data of names is sorted according to their homographs. The database has the following structure:Each set of homographs with their corresponding equivalents in Devanagari is separated out from the next set by a hard return. An example will make this... (12 Replies)
Discussion started by: gimley
12 Replies

8. Shell Programming and Scripting

Splitting a file into chunks of 1TB

Hi I have a file with different filesystems with there sizes. I need to split them in chucks of 1TB. The file looks like vf_MTLHQNASF07_Wkgp2 187428400 10601AW1 vf_MTLHQNASF07_Wkgp2 479504596 10604AW1 vf_MTLHQNASF07_Wkgp2 19940 10605AID vf_MTLHQNASF07_Wkgp2 1242622044... (4 Replies)
Discussion started by: bombcan
4 Replies

9. Shell Programming and Scripting

Deleting duplicated chunks in a file using awk/sed

Hi all, I'd always appreciate all helps from this site. I would like to delete duplicated chunks of strings on the same row(?). One chunk is comprised of four lines such as: path name starting point ending point voltage number I would like to delete duplicated chunks on the same... (5 Replies)
Discussion started by: jypark22
5 Replies

10. Shell Programming and Scripting

How to remove the text between all curly brackets from text file?

Hello experts, I have a text file with lot of curly brackets (both opening { & closing } ). I need to delete them alongwith the text between opening & closing brackets' pair. For ex: Input:- 59. Rh1 Qe4 {(Qf5-e4 Qd8-g8+ Kg6-f5 Qg8-h7+ Kf5-e5 Qh7-e7+ Ke5-f5 Qe7-d7+ Qe4-e6 Qd7-h7+ Qe6-g6... (6 Replies)
Discussion started by: prvnrk
6 Replies
squid_ldap_auth(8)					      System Manager's Manual						squid_ldap_auth(8)

NAME
squid_ldap_auth - Squid LDAP authentication helper SYNOPSIS
squid_ldap_auth -b "base DN" [-u attribute] [options] [ldap_server_name[:port]]...] squid_ldap_auth -b "base DN" -f "LDAP search filter" [options] [ldap_server_name[:port]...] DESCRIPTION
This helper allows Squid to connect to a LDAP directory to validate the user name and password of Basic HTTP authentication. The program has two major modes of operation. In the default mode of operation the users DN is constructed using the base DN and user attribute. In the other mode of operation a search filter is used to locate valid user DN's below the base DN. -b basedn (REQUIRED) Specifies the base DN under which the users are located. -f filter LDAP search filter to locate the user DN. Required if the users are in a hierarchy below the base DN, or if the login name is not what builds the user specific part of the users DN. The search filter can contain up to 15 occurrences of %s which will be replaced by the username, as in "uid=%s" for RFC2037 directo- ries. For a detailed description of LDAP search filter syntax see RFC2254. -u userattr Specifies the name of the DN attribute that contains the username/login. Combined with the base DN to construct the users DN when no search filter is specified (-f option). Defaults to 'uid' Note: This can only be done if all your users are located directly under the same position in the LDAP tree and the login name is used for naming each user object. If your LDAP tree does not match these criterias or if you want to filter who are valid users then you need to use a search filter to search for your users DN (-f option). -s base|one|sub search scope when performing user DN searches specified by the -f option. Defaults to 'sub'. base object only, one level below the base object or subtree below the base object -D binddn -w password The DN and password to bind as while performing searches. Required by the -f flag if the directory does not allow anonymous searches. As the password needs to be printed in plain text in your Squid configuration it is strongly recommended to use a account with mini- mal associated privileges. This to limit the damage in case someone could get hold of a copy of your Squid configuration file. -P Use a persistent LDAP connection. Normally the LDAP connection is only open while validating a username to preserve resources at the LDAP server. This option causes the LDAP connection to be kept open, allowing it to be reused for further user validations. Recom- mended for larger installations. -R do not follow referrals -a never|always|search|find when to dereference aliases. Defaults to 'never' never dereference aliases (default), always dereference aliases, only while searching or only to find the base object -h ldapserver Specify the LDAP server to connect to -p ldapport Specify an alternate TCP port where the ldap server is listening if other than the default LDAP port 389. EXAMPLES
For directories using the RFC2307 layout with a single domain, all you need to specify is usually the base DN under where your users are located and the server name: squid_ldap_auth -b ou=people,dc=your,dc=domain ldapserver If you have sub-domains then you need to use a search filter approach to locate your user DNs as these can no longer be constructed direcly from the base DN and login name alone: squid_ldap_auth -b dc=your,dc=domain -f uid=%s ldapserver And similarily if you only want to allow access to users having a specific attribute squid_ldap_auth -b dc=your,dc=domain -f (&(uid=%s)(specialattribute=value)) ldapserver Or if the user attribute of the user DN is "cn" instead of "uid" and you do not want to have to search for the users then you could use something like the following example for Active Directory: squid_ldap_auth -u cn -b cn=Users,dc=your,dc=domain ldapserver If you want to search for the user DN and your directory does not allow anonymous searches then you must also use the -D and -w flags to specify a user DN and password to log in as to perform the searches, as in the following complex Active Directory example squid_ldap_auth -p -R -b dc=your,dc=domain -D cn=squid,cn=users,dc=your,dc=domain -w secretsquidpassword -f (&(userPrincipal- Name=%s)(objectClass=Person)) activedirectoryserver NOTES
When constructing search filters it is strongly recommended to test the filter using ldapsearch before you attempt to use squid_ldap_auth. This to verify that the filter matches what you expect. AUTHOR
This manual page was written by Henrik Nordstrom <hno@squid-cache.org> squid_ldap_auth is written by Glenn Newton <gnewton@wapiti.cisti.nrc.ca> and Henrik Nordstrom <hno@squid-cache.org> KNOWN ISSUES
Will crash if other % values than %s is used in -f, or if more than 15 %s is used. QUESTIONS
Any questions on usage can be sent to Squid Users <squid-users@squid-cache.org>, or to your favorite LDAP list/friend if the question is more related to LDAP than Squid. REPORTING BUGS
Report bugs or bug-fixes to Squid Bugs <squid-bugs@squid-cache.org> or ideas for new improvements to Squid Developers <squid-dev@squid- cache.org> SEE ALSO
ldapsearch(1), Your favorite LDAP documentation RFC2254 - The String Representation of LDAP Search Filters, Squid LDAP Auth 25 September 2001 squid_ldap_auth(8)
All times are GMT -4. The time now is 11:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy