No, pressing button every time doesn't effect but its time consuming and if you will keep using it, there are chances during driving on D, you want to switch it on S and by mistakenly lever has gone to B.
So you shouldn't press button while you are shifting between S,D and N.