Jump to content

midi latency at recording, not the average problem


Recommended Posts

Here is yet another experiment that verifies the idea that I have about how logic works. In this example I haven't even pressed the recording button and measured the time difference afterwards. In this example I trigger my nord stage from midi notes being sent through logic. The monitoring of the Nord is through direct monitoring on my sound interface. Logics metronome is on and plays in time with the midi 4/4 notes.

 

 

I've pushed the time that Logic uses to convert the project to sound, d1, to be very large by setting buffer size to 1024 AND putting on a adaptive limiter. The adaptive limiters "look ahead" is gradually increased. We can clearly hear that the nord plays before the sound of logics metronome.

Link to comment
Share on other sites

About the experiment conducted to "loop back" the audio and midi. Here is how it looks in a diagram of the same style I posted earlier:

5mAWheH.jpg

 

*The midi is being sent at time t and gets back to the audio interface at time t resulting in almost no time difference between the midi that was sent and the midi that was recorded.

*The audio from Ultrabeat or bounced version of Ultrabeat is sent from Logic at time t, reaches the speakers at time t + d1. Also the sound of the Logic metronome arrives at time t+d1 so it's in sync.

* The audio are finished converted back to digital information at t + d1 +d2

* At arrival, Logic subtracts d1 + d2 so that the audio region on the grid ends up at t +d1 +d2 - (d1 +d2) = t therefore the audio is in sync (almost).

Link to comment
Share on other sites

So I invested a few hours into some rigorous testing and I must say the results are weird.

 

Setup: iMac 27" 2019, 3 Ghz i5, 10.14.6, Logic Pro X 10.5.1

MotU 828x

brown DX7, connected via MIDI to the 828x; MIDI loopback from DX7 THRU to 828x

(DX7 has hardware MIDI THRU)

 

The setup for all tests was as follows:

one MIDI track with manually entered notes

MIDI connection from the MotU 828x to the DX7 MIDI IN

MIDI connection from the DX7 MIDI THRU to the MotU 828x

Audio connection from DX7 to MotU 828x

project rate was 44.1

 

tempo for all tests was 120

 

This setup allows for comparatively precise measurements: The DX7 has a hardware MIDI THRU so the MIDI signal is coming back immediately without any processing. This gives us a clean timing reference both in relation to the internal clock and also to the sound sent from the DX (because the sound can never come before the MIDI data).

 

results:

buffer 1024: MIDI is spot on, Audio is about 40 ticks early

buffer 512: MIDI is spot on, Audio is about 18 ticks early

buffer 256: MIDI is spot on, Audio is about 4 ticks early

buffer 128: MIDI is spot on, Audio is about 2 ticks late

buffer 64: MIDI is spot on, Audio is about 3 ticks late

buffer 32: MIDI is spot on, Audio is about 6 ticks late

 

My preliminary conclusion is that Logic overcompensates massively the larger the buffer gets. The "sweet spot" seems to be around 128 to 64 samples buffer size; I would expect a small latency inside the DX7 as well so maybe the 32 samples buffer is closer to the real situation. This could only be measured by an oscilloscope (to check how long the DX takes to react to MIDI).

MIDI in this setup was rock solid; maximum deviation for the loopback was 1 tick for all cases.

 

I've calculated a "recording delay" of around 917 samples for the 1024 samples buffer size. This worked well for 60, 120, 240 BPM, with higher tempos (360, 480 BPM) things started to fall apart (significant MIDI jitter up to 15 ticks, Audio early up to 100 ticks!). No idea what happens with material that has rapid runs and so on.

 

Since I was interested what happens with other programs I also checked with Ableton Live 10, no driver error compensation.

 

buffer 1024: MIDI is 6x 1/256 late, Audio is 7x 1/256 late

buffer 512: MIDI is 3x 1/256 late, Audio is 4x 1/256 late

buffer 256: MIDI is less than 2x 1/256 late, Audio is less than 3x 1/256 late

buffer 128: MIDI is 1x 1/256 late, Audio is less than 2x 1/256 late

buffer 64: MIDI is less than 1x 1/256 late, Audio is more than 1x 1/256 late

buffer 32: MIDI is less than 1x 1/256 late, Audio is slightly more than 1x 1/256 late

 

Preliminary conclusions: For all buffer settings in Ableton Live the audio data arrived slightly after the loopback MIDI. The timing difference between MIDI and audio stays constant. The latency for large buffer sizes is quite high; and I'm somewhat surprised that it seems to affect MIDI and audio in an equal way.

I have no idea if the delay mostly occurs on the sending or receiving side - or perhaps on both. It's somewhat difficult to make precise measurements in Ableton Live because the exact location of events isn't displayed. So pocket calculator to the rescue: What's the length of one 1/256 at tempo 120?

1/4: 500 ms

1/8: 250 ms

1/16: 125 ms

1/32: 62.5 ms

1/64: 31.25 ms

1/128: 15.625 ms

1/256: 7.8125 ms.

 

So the latency at a buffer size of 1024 is around 47 ms for MIDI and 55 ms for audio.

 

A test for driver error resulted in a 2.8 ms error. Tests with the appropriate offset didn't show much difference which doesn't come as much of a surprise to me: The error is quite small and won't make much of a difference for larger buffer sizes anyway. For smaller buffer sizes there was a slight improvement.

 

And last but not least: Cubase 10.5

I did not run detailed tests here, mostly because I'm not familiar enough with it. A small test with a buffer size of 1024 seemed to look similar to Ableton Live. I was somewhat irritated that the list view for the MIDI data showed me notes that were completely on spot - but the arrangement view clearly showed the notes being late... Perhaps I just overlooked some setting.

 

Anyway, that's all for now. I would be interested if those things could be reproduced. The overcompensation situation looks like a possible bug to me.

Link to comment
Share on other sites

This topic comes up every once in a while do some searching to see what has been written in the past. What I can say is that LogicPro most definitely does set the record offset of midi events as you record them, but it not be doing it at the time you would like it to, particularly because you are trying to use direct monitoring.

 

I don't really want to get into the weeds trying to explain the details on this, but what I can say is that in the past I spent a lot of time trying to figure it out and I remember a few observations.

 

  1. LogicPro places the midi notes at the point in time where it figures that you heard it, after latency of the sound. If you are using a software instrument, it knows there is some latency between when you hit the note, and going to the instrument, through plugins and out the DA converter...and it will assign the midi event to the track to line up with when it thinks you actually heard that sound relative to the click you also heard. it does NOT place the midi at the point in time where it detected that you hit the actual midi note on your keyboard. It assumes that people will naturally adjust their playing to play ahead of the beat if necessary for the latency, etc.. and will place the midi event at the point in time where it calculates that you heard the sound from that midi event.
     
     
  2. It handles external instrument plugin and direct external midi tracks in different ways for this aspect. The assumption for external midi tracks is that in those old days people would be monitoring their external keyboard through an external mixer (not in the logic Mixer)...so if you use an external midi track to direct external keyboards...then the timing will actually be more in line with when you actually struck the key. LogicPro is basically going to assume there is no latency with that external keyboard, and you're monitoring it in a latency-free external mixer. But the external instrument plugin has some latency according to whatever is happening in the the mixer PDC and the assumption is that the audio is being fed back into the LogicPro mixer...with latency, etc..so then the midi event will be registered in the track, as I said...at the point of time LogicPro calculates that you heard sound.
     
     
  3. Software monitoring vs direct monitoring has different results, which I can't remember now. I while back there were some discussions about setting up INPUT buses and different things, and handling with and without software monitoring, and honestly I can't remember what was determined, but there might be a way to play with those things in order to use direct monitoring and get a tighter midi recording, that records the midi events exactly when you play it, rather then when LogicPro thinks you heard it.

Link to comment
Share on other sites

@Dewdman42 about number 2 on the list. My theory is that Logic records the midi when you press the key, almost no latency and no latency compensation. That means that if what the musician reacts to is delayed, which it is because of digital to audio conversation taking time, then the midi will sit late on the midi region. Is this what you are saying also? I can't quite work out if that is what you mean.
Link to comment
Share on other sites

LogicPro knows about not only plugins but also about your soundcard latency. That is why you hear the click exactly as the transport changes the beat number..because LogicPro knows that your sound card has latency so it makes the clock go later to match when you actually hear it. When you watch the clock, it should be matching exactly what you hear, after the metronome has gone through sound card latency.

 

Likewise...it knows that if you hear the sound late, due to either soundcard latency or plugin latency or both, it knows when you are hearing it and it will store the midi event into the region when you heard it, not when you played it.

 

This is not wrong behavior. it is a decision they made to handle it that way, which might not be what some people want, but that is intentional behavior. As I said, the assumption is that people will tend to adjust their playing so that the sound heard will be on the grid. Even a real piano has some inherent latency in the mechanisms. When people play piano, they play ahead of the grid so that the sounds will be on the grid. When peeople play an electric guitar, if the amp speaker is some distance away, there is latency between when you play it and when you hear it...and people will intuitively play ahead of the grid so that it sounds on the grid. Apple assumes that when you HEARD it is when in time you wanted it to be reproduced later. That is just a design decision.n. I don't necessarily agree with that either by the way, but that is how it is.

 

You might be able to find a tricky work around using external midi tracks or something to record your midi tracks without this assumption...then reassign that track later to a software instrument.

Link to comment
Share on other sites

Likewise...it knows that if you hear the sound late, due to either soundcard latency or plugin latency or both, it knows when you are hearing it and it will store the midi event into the region when you heard it, not when you played it.

 

If this were correct then that would be the solution to my problem, this is exactly what I want to happen! But this contradicts all of my experiment. Have you looked any of my youtube examples in the thread, the first one for example?

 

Or maybe I misunderstand you. If i play something that to my ears are exactly on the beat (direct monitored), will it end up exactly on the beat on the midi region? My experience is no.

Link to comment
Share on other sites

(This is in response to Dewdman42)

 

I think all this has nothing to do with the fact that Logic places audio material early compared to the same MIDI material - at least for large buffer settings. If I connect my DX7 both by audio and MIDI to Logic and play randomly a note the audio material will be ahead for large buffer settings

The tests I did were to determine if the audio is correct and MIDI delayed or if MIDI is correct and audio is "rushed". The latter seems to be the case: MIDI is always at the same position while the audio position varies compared to it.

 

I think what you're describing belongs to a different problem/feature set.

Link to comment
Share on other sites

One thing should have become more clear: The problem seems NOT to be that Logic places the recorded MIDI data too late; it seems more as if Logic "overcompensates" the latency for incoming audio.

 

I don't understand how you can say that logic over compensates the latency for audio. Both you and me have made test to conclude that an audio signal sent from logic and back sits just about perfectly in the same place where it came from. To me its clear that it is the other way around, the lack of compensation for midi input. It has a function for compensation of midi output, but no compensation for input.

 

EDIT: The compensation on midi output you have to tweak manually

Link to comment
Share on other sites

One thing should have become more clear: The problem seems NOT to be that Logic places the recorded MIDI data too late; it seems more as if Logic "overcompensates" the latency for incoming audio.

 

I don't understand how you can say that logic over compensates the latency for audio. Both you and me have made test to conclude that an audio signal sent from logic and back sits just about perfectly in the same place where it came from. To me its clear that it is the other way around, the lack of compensation for midi input. It has a function for compensation of midi output, but no compensation for input.

 

Well, as you know I've made also a test with an external synth where the MIDI signal sits perfectly where it should be and the audio signal is early...

And none of my tests showed MIDI being late.

 

I'm starting to think again this might be a driver issue. Both tests were done on different systems with different audio interfaces and so on.

Link to comment
Share on other sites

How are you determining if the midi is early or late? By your sense that you think you are playing exactly to the click? Just saying’

 

I already told you I don’t wanna get into the weeds on this. There have been other discussions in the past in this forum and I spent a lot of hours testing it out, not interesed in doing it again. I’ve already told you my conclusion. The matter is made more complicated by the fact that you have different modes with external tracks vs external instrument vs software Instrument, software monitoring turned on or off and if there are aux channels involved, etc If you don’t agree with me then let us know when you figure it out.

 

Or get a low latency sound card and simplify your recording process

Link to comment
Share on other sites

ps - for me this is part of why in invested in a soundcard with ultra low latency so that I basically don't have to worry about it.

 

I have a focusrite scarlett 18i20 which you can use to record in 32 buffer size with only a couple of ms of delay as a result. The problem is that it increases the load on my computer, it can handle it but the fans starts to ramp up which is very annoying. AND, as I understand it there could be a solution to this without having to record at low buffer sizes. This is of course when using hardware synths, if you use software synths you have to lower the buffer size.

Link to comment
Share on other sites

How are you determining if the midi is early or late? By your sense that you think you are playing exactly to the click? Just saying’

 

If you play (manually) an external synth and record at the same time both MIDI and audio coming from it you can at least determine where both sit in relation to each other. No click needed for that at all.

Link to comment
Share on other sites

Were you using external midi tracks or a software Inst track with the ext Inst plugin on it? Software monitoring on or off?

 

I'm not at work right now so I only have my home setup (which was used for the first test).

So I quickly did another test:

External MIDI track, software monitoring off.

Synthesizer used: Roland D-50.

Result: with a buffer size of 1024 audio is around 35 ticks early in relation to MIDI; with a buffer size of 64 it is around 9 ticks late.

Link to comment
Share on other sites

Why do you have software monitoring off while you are listening to the audio ran through the logicpro mixer?

 

Secondly, as I said already, if you use external midi tracks then logicpro handles it differently. Read this carefully: with an external midi track the assumption is that you are monitoring the sound through an external mixer and the audio is not being routed back into the latent logicpro software mixer.

Link to comment
Share on other sites

Dewdman42: Still not sure we are talking about the same thing, sorry. I tried recording the midi to software instrument track, software instrument track with the external instrument plugin, external instrument track and external instrument track with external instrument plugin. When I used the external instrument plugin i tried with both the latency compensation button on and off. Made no difference what so ever. If the midi is played externally it comes in before the audio when using higher buffer settings, always.

 

EDIT: "external instrument track" should be "external midi track"

Edited by mrcarlzone
Link to comment
Share on other sites

external instrument track and external instrument track with external instrument plugin.

 

This is not even possible....so I'm not sure what you actually did.

 

When I used the external instrument plugin i tried with both the latency compensation button on and off. Made no difference what so ever. If the midi is played externally it comes in before the audio when using higher buffer settings, always.

 

Did you turn software monitoring back on?

Link to comment
Share on other sites

Anyway, my patience on this topic is about used up, so good luck to you, let us know if you can figure it out. Generally what I can tell you is that LogicPro has more flexibility then most DAW's in terms of how to turn on and off latency adjustment capabilities. This can sometimes lead to confusion. As I said earlier, the best approach is to SIMPLIFY, and use the smallest buffer setting you can and get latent plugins off your mix while you're tracking. Latency is a fact of life...there is no magic bullet. most people are using LogicPro without any problems, usually when this kind of issue comes up its when someone is trying to do something tricky and they simply do not understand all those variables that Logic Pro provides. There are several different PDC modes, and contrary to what you just said, I can assure you that external instrument plugin handles latency and record offset different then when you use an external instrument track. then you have Low latency mode and software monitoring...and the wya plugins are handled in AUX channels vs Audio channels..etc.. these are all complicated factors.

 

There are many threads on this forum about people trying to record external instruments. I suggest you search and see what has already been said a number of times.

 

if all else fails, try cubase.

 

good luck

Link to comment
Share on other sites

We have tested different situations in this thread, you can divide them into

1. midi and audio being recorded from the "outside".

2. Midi being sent from logic and triggered audio recorded

 

For the case of number two there is a setting in the external instruments track where you can choose either "delay in ticks", "delay in ms" or "auto delay". If you choose auto delay that solves the problem of recording the audio:

 

To me that's equivalent to setting a manual midi delay except that this one is calculated automatically. And if you want to send clock to an external device then that clock is not delayed so you still have to fiddle with manual settings anyway.

 

BUT, this doesn't change the main issue I have had from the start which is midi imputed from a midi keyboard while recording, are not ending up in the right place.

Link to comment
Share on other sites

external instrument track and external instrument track with external instrument plugin.

 

This is not even possible....so I'm not sure what you actually did.

 

When I used the external instrument plugin i tried with both the latency compensation button on and off. Made no difference what so ever. If the midi is played externally it comes in before the audio when using higher buffer settings, always.

 

Did you turn software monitoring back on?

 

Still, no I never use software monitoring.

 

And where I write external instrument track it was suppose to be external MIDI track, I'm sorry

Link to comment
Share on other sites

External midi track cannot be combined with the external instrument plugin.

 

External midi track is not going to give you the desired result. External midi track assumes you are monitoring you external keyboard outside of LogicPro.

 

The external instrument plugin sits in the mixer on an instrument track, and is latency-aware.

 

When you have a high buffer setting, LogicPro knows that the incoming audio from your external keyboard is going to take a long time to get through the interface into the track, that is why it registers is early on the track so that when you play it back it should play it back at the moment in time when it believes you heard it while you were recording, with software monitoring off...you heard it through the external device in zero latency, but LogicPro still knows it is going to take a large buffer worth of time to receive it...so it registered that audio early on the track to match that amount.

 

Because you are using an external midi track and software monitoring off...your painted midi events are being sent out with an assumption that they will go to some external synth..which may or may not have some latency of its own due to midi cables, etc. but anyway, its assuming you are hearing it out there latency free...but it makes no assumption that you are recording it on another track now. The audio track you're recording is completely seperate, from LogicPro's understanding..its just audio that is coming in and it knows it takes a long time to come in through the interface, so it will register the audio early in the region to match that.

 

This is part of why external midi tracks are not good for this use. This part of why Apple introduced the external instrument plugin. the external instrument plugin sits in the mixer...rather than the environment. its a mixer thing. Because of that it is aware of plugin latency and PDC. And LogicPro is aware of PDC combined with sound card. It is specifically designed for this task of creating midi regions (on an instrument track)...which goes out to external keyboard and comes back into the mixer via the plugin itself...with latency compensation....although depending on how much latency your actual external synth adds via midi cables, interfaces, etc...that still might not be quite right, but it will be close.

 

In order to capture that as audio region, you could buss the output from that instrument channel to an audio track for recording. When you use a separate audio track, as appears to be the case, then the audio is being recorded earlier according to the sound card latency..which is exactly what we want to happen generally.

Link to comment
Share on other sites

Dewdman42, listening to you it seems that Logic has all the features that I need when you write about it. It's a real shame that I can't get it to work. I would give anything to see a real demo of how you get good sync between recorded audio and midi. Or for you to give me a link to one of those threads you talk about where this is discussed. Believe me I have googled this a lot! It would be really sad to leave this thread like this with a lot of energy put in to it but almost no result. Aspecially considering it looks like you know how to do it right? A screen recording of how it is done would be awesome. Or a very detailed instruction of how to do it.
Link to comment
Share on other sites

External midi track is not going to give you the desired result. External midi track assumes you are monitoring you external keyboard outside of LogicPro.

I have been trying to tell you, I don't use software monitoring at all. I listen to my synths directly through the sound interface, before the sound goes into the DAW.

 

The external instrument plugin sits in the mixer on an instrument track, and is latency-aware.

 

When you have a high buffer setting, LogicPro knows that the incoming audio from your external keyboard is going to take a long time to get through the interface into the track, that is why it registers is early on the track so that when you play it back it should play it back at the moment in time when it believes you heard it while you were recording, with software monitoring off...you heard it through the external device in zero latency, but LogicPro still knows it is going to take a large buffer worth of time to receive it...so it registered that audio early on the track to match that amount.

From I can understand you say there is a recording latency method that makes audio sit just right on the grid after a recording, no matter how high the buffer size is. I agree it's awesome!

 

Because you are using an external midi track and software monitoring off...your painted midi events are being sent out with an assumption that they will go to some external synth..which may or may not have some latency of its own due to midi cables, etc. but anyway, its assuming you are hearing it out there latency free...but it makes no assumption that you are recording it on another track now. The audio track you're recording is completely seperate, from LogicPro's understanding..its just audio that is coming in and it knows it takes a long time to come in through the interface, so it will register the audio early in the region to match that.

 

I understand, it compensates all audio coming in with d1 and d2 summed up, where d1 is the delay going out of the computer and d2 the delay coming in. That's great, I have no issues with that. Maybe the user gacki has some issues with that, he have mentioned he thinks logic overcompensates but I don't agree.

 

This is part of why external midi tracks are not good for this use. This part of why Apple introduced the external instrument plugin. the external instrument plugin sits in the mixer...rather than the environment. its a mixer thing. Because of that it is aware of plugin latency and PDC. And LogicPro is aware of PDC combined with sound card. It is specifically designed for this task of creating midi regions (on an instrument track)...which goes out to external keyboard and comes back into the mixer via the plugin itself...with latency compensation....although depending on how much latency your actual external synth adds via midi cables, interfaces, etc...that still might not be quite right, but it will be close.

 

In order to capture that as audio region, you could buss the output from that instrument channel to an audio track for recording. When you use a separate audio track, as appears to be the case, then the audio is being recorded earlier according to the sound card latency..which is exactly what we want to happen generally.

 

Ok, I try create a software instrument track with en external instrument plugin and bus the audio out to another audio channel. I record those two channels at the same time and I still have the same problem, here:

 

Can you please watch it and point out what I'm doing wrong? Feels like we're a bit closer now to a solution.

Link to comment
Share on other sites

When you have a high buffer setting, LogicPro knows that the incoming audio from your external keyboard is going to take a long time to get through the interface into the track, that is why it registers is early on the track so that when you play it back it should play it back at the moment in time when it believes you heard it while you were recording, with software monitoring off...you heard it through the external device in zero latency, but LogicPro still knows it is going to take a large buffer worth of time to receive it...so it registered that audio early on the track to match that amount.

 

I get what you're saying but I don't understand the reasoning behind it. If it really is as you describe it it would be a huge PITA because it would also open up a whole can of worms.

A few examples/questions.

Why would Logic not put the audio material directly ON the grid and then negatively delay it only during playback?

The way you describe it we would get different timing when recording with one buffer setting and playback with another one. This is especially worrying because this is a typical recommendation: record with small buffer settings, mix and playback with larger ones (to free up processing power).

The timing between MIDI regions and audio regions would also change when varying the buffer size.

What about drummer tracks? Doing a "bounce in place" on them doesn't put them early; the transients line up with the MIDI data (and the grid). Wouldn't "bounce in place" have to account for those different buffer sizes?

What about Loops from the library - wouldn't we have to put them early as well?

 

Now, for me it's more or less an academic question because I wouldn't record with a buffer setting higher than 128 anyway. But often such problems with extreme settings shine a light on a hidden bug or design flaw.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...