Network Access VPN and host networks (VirtualBox)

We have a Network Access VPN with Full Access (i.e: All traffic goes through the VPN) I've noticed that with this configuration, even the traffic from the host to locally hosted VMs (on VirtualBox) gets routed trough the tunnel when the VPN is up which is not desired. One usually wants/expects to have the VPN on but can go on with working on his locally installed/running Virtual Machines

(I'm not talking here about the network traffic from withing the VM to the outside world, I'm only interested in the traffic from the host to the VM)

I was wondering if is there any subtle trick to let the VPN client ignore this kind of traffic (maybe the interfaces) and let it flow outside the tunnel (without flipping the overall conf of the Network Access to "split tunnel")

Youssef Ghorbal