I'm not used to *BSD, but in linux, when you try to load a kernel moule with "modprobe" (which I think is equivalent to kld[un]load), you must not use the module's file name. Instead, just its name.
Try to "kldunload if_my"
Anyway, I agree sysgate. A priori, there's no need to do so IMO.
Please, post the output of:
ifconfig -a
netstat -r

Regards.