Performance comparsion with Bidule

[torsig1967]

EDIT: PLEASE READ THE WHOLE THREAD TO GET THE FULL PICTURE, OTHERWISE YOU WILL BE MISINFORMED SINCE MY INITIAL ASSUMPTIONS WERE WRONG!

 

I compared Mainstage and Bidule. Sending three channels of seuence data from Numerology (virtual midi routing) to each application. With three audio unit instruments loaded (one for each channel).

 

No other effects.

Mainstage/Bidule tested separately, not running at the same time.

 

CPU perantage according to activity monitor:

Buffersize=128

Mainstage: 82%

Bidule: 45%

 

Buffersize=256

Mainstage: 73%

Bidule: 39%

 

That's quite a big difference.

I wonder why, is it the graphics?

 

Mac OS 10.4.11, MacBookPro Core 2 duo 2.2 GHz.

Edited May 9, 2008 by torsig1967

[mmm42]

I compared Mainstage and Bidule. Sending three channels of seuence data from Numerology (virtual midi routing) to each application. With three audio unit instruments loaded (one for each channel).

 

No it is probably not the graphics. But it is a totally different audio engine... MainStage has a couple of special features, which you don't test with such a simple test.

 

If you just want to use 3 AUs without a context, then use AULab, part of the developer tools and free...

[torsig1967]

No it is probably not the graphics. But it is a totally different audio engine... MainStage has a couple of special features, which you don't test with such a simple test.

 

If you just want to use 3 AUs without a context, then use AULab, part of the developer tools and free...

 

But which are those special features? It would be interesting to compare using those and mimic similar behavior in Bidule and see if the difference is smaller.

 

I find the user interface and looks of Mainstage very well designed and it's fun too, but that doesn't explain the need for so much CPU. I really don't see why Mainstage need so much more than the other hosts I tried. It doesn't even have a timeline.

 

Yes if I install developer tools it would be interesting to compare AUlab with Bidule which so far has the best CPU-performance of the hosts I've tried.

[mmm42]

CPU perantage according to activity monitor:

 

Ok, there is your issue: "Activity Monitor" is not a tool to measure real-time applications. It only shows you the load on the CPU(s). But it says _nothing_ about performance of an application! Even if Activity Monitor would show 100% load, you can probably still add more plugins to a channel strip - it would only slow down your graphics.

 

If you want to know the difference in the audio engine, you have to setup identical channel strips in both applications and then duplicate them till the system can no longer handle them. The number of channel strips is your performance measurement. Of course the number should be reasonably high, otherwise your measurement error is too large.

[torsig1967]

CPU perantage according to activity monitor:

 

Ok, there is your issue: "Activity Monitor" is not a tool to measure real-time applications. It only shows you the load on the CPU(s). But it says _nothing_ about performance of an application! Even if Activity Monitor would show 100% load, you can probably still add more plugins to a channel strip - it would only slow down your graphics.

 

If you want to know the difference in the audio engine, you have to setup identical channel strips in both applications and then duplicate them till the system can no longer handle them. The number of channel strips is your performance measurement. Of course the number should be reasonably high, otherwise your measurement error is too large.

Yes, that's a good point. I'll try something like that for comparsion.

 

Still there's good reason to compare Activity Monitor/CPU consumption also. If one is planning to run several applications simultaneously, there's a potential problem with applications that by default uses a lot of CPU time leaving less to the others.

[mmm42]

Still there's good reason to compare Activity Monitor/CPU consumption also. If one is planning to run several applications simultaneously, there's a potential problem with applications that by default uses a lot of CPU time leaving less to the others.

 

If they are all real-time applications: no.

[torsig1967]

Still there's good reason to compare Activity Monitor/CPU consumption also. If one is planning to run several applications simultaneously, there's a potential problem with applications that by default uses a lot of CPU time leaving less to the others.

 

If they are all real-time applications: no.

 

How do you mean?

 

Let's say I only want to use one instance of EXS24 in Mainstage but a lot of other stuff in Bidule. Mainstage would "overuse" the CPU in this case, leaving less than necessary for Bidule.

[mmm42]

Still there's good reason to compare Activity Monitor/CPU consumption also. If one is planning to run several applications simultaneously, there's a potential problem with applications that by default uses a lot of CPU time leaving less to the others.

 

If they are all real-time applications: no.

 

How do you mean?

 

Let's say I only want to use one instance of EXS24 in Mainstage but a lot of other stuff in Bidule. Mainstage would "overuse" the CPU in this case, leaving less than necessary for Bidule.

 

Activitity Monitor does not show how much real-time CPU applications use. Any non-realtime CPU load doesn't affect playback or recording.

[torsig1967]

Still there's good reason to compare Activity Monitor/CPU consumption also. If one is planning to run several applications simultaneously, there's a potential problem with applications that by default uses a lot of CPU time leaving less to the others.

 

If they are all real-time applications: no.

 

How do you mean?

 

Let's say I only want to use one instance of EXS24 in Mainstage but a lot of other stuff in Bidule. Mainstage would "overuse" the CPU in this case, leaving less than necessary for Bidule.

 

Activitity Monitor does not show how much real-time CPU applications use. Any non-realtime CPU load doesn't affect playback or recording.

 

OK, seems I miss something here. I really don't understand what Activity Monitor shows me in that case. Since it's updated all the time I thought it showed real time (what's happening at the moment) usage.

 

Maybe you can point me the direction to some document covering this area so I can get a better understanding of it?

[mmm42]

OK, seems I miss something here. I really don't understand what Activity Monitor shows me in that case. Since it's updated all the time I thought it showed real time (what's happening at the moment) usage.

 

Quite simple: don't use it. It measures the work your CPU(s) are currently doing, but this is NOT what YOU want to measure. You want to know how much MIDI/Audio work you applications can do.

 

Mac OS X has different priorities for different operations. Most of the operations have a quite low priority (Spotlight, Applications like TextEdit, Word, and so on), but a few Applications with real-time needs (MIDI/Audio/Video that has to be recorded/played in a guaranteed timely manner) have a much higher priority.

 

Even if your system is at a 100% load because of Word, you still can use probably 99% more of the system for real-time applications. Word will come to a crawling stop, but that's to be expected. Activity Monitor would continuously show 100% CPU load over the whole time (first with Word, then with e.G. Logic). It always shows the sum of all activity. To make it more tricky: an application can have several priority levels for different parts of the application. An example is e.g. graphics in MainStage, which actually runs with a lower priority, while the Audio/MIDI part runs with a high priority. Activity Monitor shows the sum of both, which is useless for any estimate. The CPU load inside MainStage is better, but still an estimate (the actual load is impossible to measure)

[torsig1967]

Well, it's absolutely clear that my initial way of testing was not correct.

 

Now I've tested both Bidule and Mainstage playing back midi sent from Numerology and adding channelstrips one by one hearing when the sound out starts to distort, click and go bad.

 

This test shows that Mainstage and Bidule have quite a similar performance. Both could play four of my channelstrips at the same time. There might have been some very small clicks out of Bidule even with four strips which indicates that Mainstage have a bit better performance.

 

The channelstrip consisted of:

impOscar/MonstaChorus/CamelCrusher/PingPongDelay/AUMAtrixreverb/Luxonix1310.

 

CONCLUSION: There's no big difference in cpu-performance between Mainstage and Bidule.

 

(Which is great news for me since I want to use Mainstage for all "normal" kind of live playing.)

[torsig1967]

I also just found that when running Bidule and Mainstage at the same time I can play 3 channel strips each at the same time. That's a total of six, two more than when running only one application!

 

That's also good news for anyone planning to run more than Mainstage at the same time.