It depends on the party/person scanning you...

For example your ISP may port scan you, which is legal