|
|
Sponsored Content
Special Forums
Cybersecurity
IT Security RSS
AMTSO: Testing Standards Revisited
Post 302253849 by Linux Bot on Sunday 2nd of November 2008 09:40:09 PM
11-02-2008
AMTSO: Testing Standards Revisited
It's been a long hard few months in the anti-malware industry (which is why I haven't blogged outside work for a while): for some reason, all our conferences, workshops etc. all seem to be jammed into the last few months of the year. One event I'm always glad to get to is Virus Bulletin, the premier conference and networking opportunity for people in my speciality, but that was a month ago, so I guess it's a bit late to blog about it. I've just got back, though, from a meeting of AMTSO (Anti-Malware Testing Standards Organization), and that has me feeling more positive about the state of anti-malware testing than I have in quite a while.
Product testing (and especially detection testing) is the bete noire of the anti-malware industry. Once upon a time, when the threat landscape was a lot populated than today, it was all a little less fraught. If you found your product credited with a near-zero detection rate, there was a chance, if you managed to establish contact with the tester, to find out what was really happening.
Now, though, when anti-virus labs routinely receive 100,000 or more unique samples a day and we tend to assume a margin for error of +/- 10% to allow for regional bias, validation errors, and so on... The problem is, we tend to find it easier to tell people what they should be doing than to advise them on how to do it properly (or what we think of as properly...) However, the AMTSO meeting represents, I think, something of a coming of age for the representatives of the anti-malware industry taking part, not to mention the testers, reviewers, publishers and so on who are also taking part.
We've been working for some time on two major documents: one on "The Fundamental Principles of Testing" and one on "Best Practices for Dynamic Testing." So it was a joy it was to have the final versions of both documents unanimously approved on the last day of the conference. Neither is going to stop bad testing, but they'll go a long way towards giving people with a genuine interest in good testing (whether as a tester or as a consumer) some of the knowledge they need if standards are to be raised across the board. This is an excellent step forward in making available a vendor-agnostic informational resource, and there are several other resources on the way. (Unfortunately, I'm going to have to write some of them...)
David Harley CISSP FBCS CITP
Director of Malware Intelligence, ESET LLC
More...
Product testing (and especially detection testing) is the bete noire of the anti-malware industry. Once upon a time, when the threat landscape was a lot populated than today, it was all a little less fraught. If you found your product credited with a near-zero detection rate, there was a chance, if you managed to establish contact with the tester, to find out what was really happening.
Now, though, when anti-virus labs routinely receive 100,000 or more unique samples a day and we tend to assume a margin for error of +/- 10% to allow for regional bias, validation errors, and so on... The problem is, we tend to find it easier to tell people what they should be doing than to advise them on how to do it properly (or what we think of as properly...) However, the AMTSO meeting represents, I think, something of a coming of age for the representatives of the anti-malware industry taking part, not to mention the testers, reviewers, publishers and so on who are also taking part.
We've been working for some time on two major documents: one on "The Fundamental Principles of Testing" and one on "Best Practices for Dynamic Testing." So it was a joy it was to have the final versions of both documents unanimously approved on the last day of the conference. Neither is going to stop bad testing, but they'll go a long way towards giving people with a genuine interest in good testing (whether as a tester or as a consumer) some of the knowledge they need if standards are to be raised across the board. This is an excellent step forward in making available a vendor-agnostic informational resource, and there are several other resources on the way. (Unfortunately, I'm going to have to write some of them...)
David Harley CISSP FBCS CITP
Director of Malware Intelligence, ESET LLC
More...
|
|
7 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a ascii file with lines like this:
240|^M\ ^M\^M\ Old Port Marketing order recd $62,664.- to ship 6/22/99^M\
when this record gets loaded into my database, the \ is stored literally and so the user sees carriage return \ (hex 0D 5C) when what i need is carriage return line feed (hex 0D... (1 Reply)
Discussion started by: Brandt
1 Replies
2. Solaris
I just installed solaris 9 on a sunblade 150(sparc), and have it partitioned. I've been using ufsrestore to restore bring the config from my old system, to the sunblade. I'm not having any luck. The root directory restore seems to work. When I try to restore /usr, I get an "/usr/sbin/fsck not... (4 Replies)
Discussion started by: ECBROWN
4 Replies
3. Solaris
in ufsrestore how do i know which volume my selected
folder or file exist?. (4 Replies)
Discussion started by: S26+
4 Replies
4. UNIX for Advanced & Expert Users
I have read through all documents in FAQ and have run into an issue with sending an email with body message text and an email attachment. I have included what I have thus far and I can get the message body to send in the email to work only. I cannot understand the uuencode even after I read the... (5 Replies)
Discussion started by: tekline
5 Replies
5. Virtualization and Cloud Computing
Many readers have read the hype, experienced the Orwellian marketspeak, watched the positioning debates, and seen poorly managed software companies play the game of analyst-chasing (similar to ambulance chasing when you think about it). Finally, the up-to-date definitions, and hopefully a bit of... (0 Replies)
Discussion started by: Linux Bot
0 Replies
6. Solaris
hi all,
ive downloaded ,built and installed coreutils from sunfreeware.com,in my quest to get the color display when ls is used(linux style)...
After the pkg is installed,how do i use ls to get the color?
I know its installed because i get a host of cmds that have been updated,l
like this,
... (1 Reply)
Discussion started by: wrapster
1 Replies
7. UNIX for Dummies Questions & Answers
I'm using this in AIX to find what file contains the value 'batch' in it, in all directories.
find / -type f -exec grep -l batch {} /dev/null \;
My question is, what if I only wanted to search *.sh files, and I wanted to pipe the results to a file called 'batch_find.txt'. How could I code... (3 Replies)
Discussion started by: NycUnxer
3 Replies
LEARN ABOUT HPUX
rblsmtpd
rblsmtpd(1) General Commands Manual rblsmtpd(1) NAME
rblsmtpd - blocks mail from RBL-listed sites. It works with any SMTP server that can run under tcpserver(1) SYNOPSIS
rblsmtpd opts prog DESCRIPTION
opts is a series of getopt-style options. prog consists of one or more arguments. Normally rblsmtpd runs prog. prog is expected to carry out an SMTP conversation to receive incoming mail messages. However, rblsmtpd does not invoke prog if it is told to block mail from this client. Instead it carries out its own limited SMTP conversa- tion, temporarily rejecting all attempts to send a message. Meanwhile it prints one line on descriptor 2 to log its activity. rblsmtpd drops the limited SMTP conversation after 60 seconds, even if the client has not quit by then. OPTIONS
-t n Change the timeout to n seconds. Blocked clients If the $RBLSMTPD environment variable is set and is nonempty, rblsmtpd blocks mail. It uses $RBLSMTPD as an error message for the client. Normally rblsmtpd runs under tcpserver(1); you can use tcprules(1) to set $RBLSMTPD for selected clients. If $RBLSMTPD is set and is empty, rblsmtpd does not block mail. If $RBLSMTPD is not set, rblsmtpd looks up $TCPREMOTEIP in the RBL, and blocks mail if $TCPREMOTEIP is listed. tcpserver sets up $TCPRE- MOTEIP as the IP address of the remote host. -r base Use base as an RBL source. An IP address a.b.c.d is listed by that source if d.c.b.a.base has a TXT record. rblsmtpd uses the con- tents of the TXT record as an error message for the client. -a base Use base as an anti-RBL source. An IP address a.b.c.d is anti-listed by that source if d.c.b.a.base has an A record. In this case rblsmtpd does not block mail. You may supply any number of -r and -a options. rblsmtpd tries each source in turn until it finds one that lists or anti-lists $TCPRE- MOTEIP. It also tries an RBL source of rbl.maps.vix.com if you do not supply any -r options. See http://maps.vix.com/rbl/ for more informa- tion about rbl.maps.vix.com. If you want to run your own RBL source or anti-RBL source for rblsmtpd, you can use rbldns from the DNScache (djbdns) package. Temporary errors Normally, if $RBLSMTPD is set, rblsmtpd uses a 451 error code in its limited SMTP conversation. This tells legitimate clients to try again later. It gives innocent relay operators a chance to see the problem, prohibit relaying, get off the RBL, and get the mail delivered. However, if $RBLSMTPD begins with a hyphen, rblsmtpd removes the hyphen and uses a 553 error code. This tells legitimate clients to bounce the message immediately. There are several error-handling options for RBL lookups: -B (Default.) Use a 451 error code for IP addresses listed in the RBL. -b Use a 553 error code for IP addresses listed in the RBL. -C (Default.) Handle RBL lookups in a ``fail-open'' mode. If an RBL lookup fails temporarily, assume that the address is not listed; if an anti-RBL lookup fails temporarily, assume that the address is anti-listed. Unfortunately, a knowledgeable attacker can force an RBL lookup or an anti-RBL lookup to fail temporarily, so that his mail is not blocked. -c Handle RBL lookups in a ``fail-closed'' mode. If an RBL lookup fails temporarily, assume that the address is listed (but use a 451 error code even with -b). If an anti-RBL lookup fails temporarily, assume that the address is not anti-listed (but use a 451 error code even if a subsequent RBL lookup succeeds with -b). Unfortunately, this sometimes delays legitimate mail. SEE ALSO
tcpserver(1), tcprules(1), tcprulescheck(1), fixcrio(1), recordio(1), rblsmtpd(1), tcpclient(1), who@(1), date@(1), finger@(1), http@(1), tcpcat(1), mconnect(1), tcp-environ(5) http://cr.yp.to/ucspi-tcp.html rblsmtpd(1)