Sponsored Content
Top Forums Shell Programming and Scripting remove a specific line in a LARGE file Post 302350888 by blubbiblubbkekz on Sunday 6th of September 2009 06:22:50 AM
Old 09-06-2009
remove a specific line in a LARGE file

Hi guys,
i have a really big file, and i want to remove a specific line.
Code:
sed -i '5d' file

This doesn't really work, it takes a lot of time...
The whole script is supposed to remove every word containing less than 5 characters and currently looks like this:
Code:
#!/bin/bash

line="1"
wholelines="38867934"
while [ $line -lt $wholelines ]; do
length=$(expr length `sed $line'q;d' list`)
if [ $length -lt "5" ]; then
sed -i $line' d' list
wholelines=$((wholelines-1))
else
line=$(($line+1))
fi
done

so is there a way to remove a specific line in less time than sed needs on large files?


Uli
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Line that contains specific string

I am looking for a way to remove any line in a text file that contains the string "Mac address". I guess you would grep and sed, but I am not sure how to do this. Thanks for you help. (3 Replies)
Discussion started by: CBarraford
3 Replies

2. UNIX for Dummies Questions & Answers

Help with selecting specific lines in a large file

Hello, I need to select the 3 lines above as well as below a search string, including the search string. I have been trying various combinations using sed command without any success. Can anuone help please. Thanking (2 Replies)
Discussion started by: tansha
2 Replies

3. Shell Programming and Scripting

Count specific character(s) very large file

I'm trying to count the number of 2 specific characters in a very large file. I'd like to avoid using gsub because its taking too long. I was thinking something like: awk '-F' { t += NF - 1 } END {print t}' infile > outfile which isn't working Any ideas would be great. (3 Replies)
Discussion started by: dcfargo
3 Replies

4. Shell Programming and Scripting

Searching a specific line in a large file

Hey All Can any one please suggest the procedure to search a part of line in a very large file in which log entries are entered with very high speed. i have trued with grep and egrep grep 'text text text' <file-name> egrep 'text text text' <file-name> here 'text text text' is... (4 Replies)
Discussion started by: NIMISH AGARWAL
4 Replies

5. Shell Programming and Scripting

remove unwanted specific line range

Hello everyone...I have large txt file and I would like to remove unwanted specific line. My data is like this: So I would like to remove from line below No. until line reassambled like this: Thanks... (4 Replies)
Discussion started by: taxi
4 Replies

6. Shell Programming and Scripting

How to remove a subset of data from a large dataset based on values on one line

Hello. I was wondering if anyone could help. I have a file containing a large table in the format: marker1 marker2 marker3 marker4 position1 position2 position3 position4 genotype1 genotype2 genotype3 genotype4 with marker being a name, position a numeric... (2 Replies)
Discussion started by: davegen
2 Replies

7. Shell Programming and Scripting

Remove line with specific character

HI Input :- Aog:0rt__dev_8 LAAXU24 vs.3 LAA40l0 ** LAAXU241 ** Output :- Aog:0rt__dev_8 LAAXU24 vs.3 Delete the line with ** (3 Replies)
Discussion started by: pareshkp
3 Replies

8. Shell Programming and Scripting

Remove every line with specific string, and also the one above and below it

I would like to identify every line with a specific string, in this case: "Mamma". I would like to remove that line, and also the line above it and below it. So the below Where are all amazing Flats Look At The Great Big White Hey There Hot Mamma You Are So hot Baby I wish You were Mine... (5 Replies)
Discussion started by: phpchick
5 Replies

9. Shell Programming and Scripting

Bash : Checking Large file for specific lines

Morning .. I have a file with approximately 1000 lines. I want to check that the file contains, for example, 100 lines. Something like whats given below is ugly. And even if I create a function I have to call it 100 times. I may need to look through multiple files at times. Is there a... (4 Replies)
Discussion started by: sumguy
4 Replies

10. Shell Programming and Scripting

Remove dupes in a large file

I have a large file 1.5 gb and want to sort the file. I used the following AWK script to do the job !x++ The script works but it is very slow and takes over an hour to do the job. I suspect this is because the file is not sorted. Any solution to speed up the AWk script or a Perl script would... (4 Replies)
Discussion started by: gimley
4 Replies
LOGPROF.CONF(5)                                                      AppArmor                                                      LOGPROF.CONF(5)

NAME
logprof.conf - configuration file for expert options that modify the behavior of the AppArmor aa-logprof(1) program. DESCRIPTION
The aa-logprof(1) program can be configured to have certain default behavior by the contents of logprof.conf. The [qualifiers] section lists specific programs that should have a subset of the full ix/px/ux list when asking what mode to execute it using. Since creating a separate profile for /bin/bash is dangerous, we can specify that for /bin/bash, only (I)nherit, (U)nconstrained, and (D)eny should be allowed options and only those will show up in the prompt when we're asking about adding that to a profile. Likewise, if someone currently exec's /bin/mount in ix or px mode, things won't work, so we can provide only (U)nconstrained and (D)eny as options. And certain apps like grep, awk, sed, cp, and mkdir should always inherit the parent profile rather than having their own profile or running unconfined, so for them we can specify that only (I)nherit and (D)eny are the allowed options. Any programs that are not listed in the qualifiers section get the full (I)nherit / (P)rofile / (U)nconstrained / (D)eny option set. If the user is doing something tricky and wants different behavior, they can tweak or remove the corresponding line in the conf file. The [defaulthat] section lists changehat-aware programs and what hat aa-logprof(1) will collapse the entries to for that program if the user specifies that the access should be allowed, but should not have it's own hat. The [globs] section allows modification of the logprof rule engine with respect to globbing suggestions that the user will be prompted with. The format of each line is-- "<perl glob> = <apparmor glob>". When aa-logprof(1) asks about a specific path, if the perl glob matches the path, it replaces the part of the path that matched with the corresponding apparmor glob and adds it to the list of globbing suggestions. Lines starting with # are comments and are ignored. EXAMPLE
[qualifiers] # things will very likely be painfully broken if bash has it's own profile /bin/bash = iu # mount doesn't work if it's confined /bin/mount = u # these helper utilities should inherit the parent profile and # shouldn't have their own profiles /bin/awk = i /bin/grep = i /bin/sed = i [defaulthat] /usr/sbin/sshd = EXEC /usr/sbin/httpd2 = DEFAULT_URI /usr/sbin/httpd2-prefork = DEFAULT_URI [globs] # /foo/bar/lib/libbaz.so -> /foo/bar/lib/lib* /lib/lib[^/]+so[^/]*$ = /lib/lib*so* # strip kernel version numbers from kernel module accesses ^/lib/modules/[^/]+/ = /lib/modules/*/ # strip pid numbers from /proc accesses ^/proc/d+/ = /proc/*/ BUGS
If you find any bugs, please report them at <http://https://bugs.launchpad.net/apparmor/+filebug>. SEE ALSO
apparmor(7), apparmor.d(5), aa-enforce(1), aa-complain(1), aa-disable(1), aa_change_hat(2), aa-logprof(1), aa-genprof(1), and <http://wiki.apparmor.net>. AppArmor 2.7.103 2012-06-28 LOGPROF.CONF(5)
All times are GMT -4. The time now is 10:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy