So basically I narrowed it down to two things!
1. A chap over on Gearslutz said he'd run into this once when, and I'll quote, "This can happen if you have two copies of the same plugin, one in the system components folder where they normally live, and one in the user version of that folder: /Users/[user name]/Library/Audio/Plug-Ins/Components' and lo and behold I found HornetVU in both folders. Not sure how it got there, maybe an installation gremlin, but deleted the instance in my User folder and I was down to four plugins still acting up.
2. The remaining plugs were all leftovers from my Mojave install that didn't seem to have the proper security signing/clearances that Apple brought in with Catalina. Somehow they could still be validated in the AU scan (not sure how?) but it was holding it up for a moment (hence the consistent rescanning on start-up) before launching.
I found that if I "aborted" the scan during launch (I had to be quick because it would only flash for a second or two) it would ask if I wanted to continue without the plugins and I would hit yes (or 'continue' or whatever it said) and then if I immediately went to the Plugin Manager I could see which plug was causing the problem.
I repeated this four times deleting each plugin and then restarting and relaunching and voila finally no rescan at launch.
R.