A technical support community for Apple Logic Pro users.

 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Tue May 18, 2021 12:55 am

mrcarlzone wrote:
While I was doing that I found something else. When I changed sound from piano to organ on my nord stage the distance between the midi and the sound transient increased from 50 to 70 ticks. It seems like the nord has a built in latency when playing piano samples. The organ sound is not samples but modelled, that could be the reason for the difference.


I had a similar thought. However this wouldn't (fully) explain why the latency is different for different buffer sizes.
There are other possible (unknown) factors. As far as I see it the Nord is connected via 5-pin-MIDI to the USB interface. It would be interesting to see if the situation is different when the Nord is connected directly via USB-MIDI to the computer (if it is a model that allows this).
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Tue May 18, 2021 4:53 am

Forget the whole "change sound from piano to organ" i wrote in my previous post.

Nord stage has a special organ trigger function which is made to be able to play organ on a weighted piano. For the piano samples to sound you have to press the keys all the way down, but for the organ to play you just have to press it slightly. If you press it gentle enough, the organ will sound but no midi is sent. If you play it normally, the organ will sound and midi information is sent 10-50 ticks later depending on time it takes for the key to travel from the top to the bottom.

All in all, we can totally ignore the "piano sound vs organ sound" thing, it's just a special feature on the nord stage and I don't want to derail the main discussion.

EDIT: I've come to this conclusion thru triggering the nord and another synth (prophet 8) from another midi controller (octatrack). When the nord stage is played from midi notes sent from the octatrack to the prophet 8 and to the nord stage via midi thru both their sounds arrive at the same time to logic.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 5:07 am

To sum up. I think we can say that this is how it works in logic if you try to record audio and midi at the same time:

1. The metronome click is played from logic at time t
2. The click is converted to audio and heard at time t+d1
3. An outboard synth is played at t + d1 which sounds right on the beat, and also sends midi at the same time
4. The audio arrives back to logic at t + d1 + d2, the midi on the other hand travels back to logic in practically no time and arrives at time t + d1
5. Logic knows the values of d1 and d2, and adjusts the audio back to t, Logic doesn't do anything to correct the midi and leaves it sitting at t+d1

Image
 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 5:42 am

No, that's not correctly describing what happens.
Logic actually gives a negative delay to all of its audio data (also the click) when playing it back so that it appears at the same time as a MIDI click. The click (and other audio material that is already present) is heard at t. That's what the latency compensation does.

As I wrote that my own "loopback" testing was "interesting": I was playing back both MIDI and audio data and re-recording it into Logic.
Result: audio was slightly late, MIDI was slightly early (!).
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 5:59 am

Interesting allthough im not sure i understand. Show it please with a screen recording or something, with buffer size set to 1024.
 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 6:46 am

Here we go:

Bildschirmfoto 2021-05-19 um 15.24.17.png
Bildschirmfoto 2021-05-19 um 15.24.17.png (350.39 KiB) Viewed 673 times


Upper Ruler is in milliseconds. We can see that all loopback audio is about 1 ms late; this is true for the metronome, the Ultrabeat click (coming directly from MIDI data) and the Ultrabeat click coming from the BiP file (the transients of this one don't line up with the other two because the transient in the BiP file is a bit late).

The two Ultrabeat tracks at the top were used for the MIDI loopback.
First track was used for pure MIDI loopback without Audio; recorded MIDI data is about 3 ms early.
Second track was used for combined audio/MIDI loopback (together with the Audio track below it); MIDI is here about 5 ms early. (this decreases later)

All tests done with an M-Audio M-Track 2x2M and a buffer size of 1024.

Preliminary conclusions: There's somewhere an audio driver error of roughly 1 ms lurking; otherwise audio loopback seems to be consistent. MIDI loopback is early and somewhat "jittery": the first notes are even "earlier" than the following. Pure MIDI loopback: the first two notes were 6 ticks early, the rest 4 ticks. Combined Audio/MIDI loopback: the first notes were 10 ticks early, the rest again 4 ticks.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 7:56 am

What outboard synth was used?
 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 8:29 am

None, it's pure loopback from Logic to Logic.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 11:27 am

Ok, so you've confirmed that recording midi directly from the output of the interface at the same time as recording audio from the output of the interface yields recordings that sits just about perfect or like 3-4 ms off. The same that I did but you did it on your hardware and conducted the test simultaneously, correct?

I post another video that confirm that my model above about audio/midi-sync seems to be correct.
https://youtu.be/XLVu-0m_45A

Here an external instrument region has 4/4 notes with midi repeating. During recording it is sent to the nord stage and record the audio. According to my understandning, the midi travels without any noticeable delay to the nord stage, arrives at time t, the nord plays the sound, that sound arrive in logic at time t+d2. Logic then continues to latency-compensate the audio track by removing d1+d2 from t+d2 which yields
t+d2 - (d1+d2)=t-d1
The audio sits before the beat by d1 ms, in this case about 25-30 ms.

EDIT: to fix this I go in the midi settings menu and add 27 ms of delay. I thought you said that Logic does this automatically, or maybe I misunderstood? After that change the recording of the audio is almost exactly on the beat.
 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 3:11 pm

I'll try to do a test on different hardware tomorrow, but in general I'd suspect a driver "problem" here.
There are different layers to latency compensation and at each step something can go wrong. Logic has to compensate for different types of latency: latency introduced by plugins and instruments and latency originating with the buffer size and audio driver.
Especially the latter can be tricky because the driver's behavior can be unpredictable (it shouldn't, though!).

One thing should have become more clear: The problem seems NOT to be that Logic places the recorded MIDI data too late; it seems more as if Logic "overcompensates" the latency for incoming audio. The solution for this problem should consequently NOT come from delaying MIDI data (which could open a completely different can of worms) but from fixing the "overcompensation".

As I wrote: I'll look into this tomorrow.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 8:02 pm

Here is yet another experiment that verifies the idea that I have about how logic works. In this example I haven't even pressed the recording button and measured the time difference afterwards. In this example I trigger my nord stage from midi notes being sent through logic. The monitoring of the Nord is through direct monitoring on my sound interface. Logics metronome is on and plays in time with the midi 4/4 notes.

https://youtu.be/oh26d31EWBU

I've pushed the time that Logic uses to convert the project to sound, d1, to be very large by setting buffer size to 1024 AND putting on a adaptive limiter. The adaptive limiters "look ahead" is gradually increased. We can clearly hear that the nord plays before the sound of logics metronome.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Wed May 19, 2021 8:16 pm

About the experiment conducted to "loop back" the audio and midi. Here is how it looks in a diagram of the same style I posted earlier:
Image

*The midi is being sent at time t and gets back to the audio interface at time t resulting in almost no time difference between the midi that was sent and the midi that was recorded.
*The audio from Ultrabeat or bounced version of Ultrabeat is sent from Logic at time t, reaches the speakers at time t + d1. Also the sound of the Logic metronome arrives at time t+d1 so it's in sync.
* The audio are finished converted back to digital information at t + d1 +d2
* At arrival, Logic subtracts d1 + d2 so that the audio region on the grid ends up at t +d1 +d2 - (d1 +d2) = t therefore the audio is in sync (almost).
 
gacki
Posts: 172
Joined: Fri Jul 17, 2020 3:18 am

Re: midi latency at recording, not the average problem

Thu May 20, 2021 8:30 am

So I invested a few hours into some rigorous testing and I must say the results are weird.

Setup: iMac 27" 2019, 3 Ghz i5, 10.14.6, Logic Pro X 10.5.1
MotU 828x
brown DX7, connected via MIDI to the 828x; MIDI loopback from DX7 THRU to 828x
(DX7 has hardware MIDI THRU)

The setup for all tests was as follows:
one MIDI track with manually entered notes
MIDI connection from the MotU 828x to the DX7 MIDI IN
MIDI connection from the DX7 MIDI THRU to the MotU 828x
Audio connection from DX7 to MotU 828x
project rate was 44.1

tempo for all tests was 120

This setup allows for comparatively precise measurements: The DX7 has a hardware MIDI THRU so the MIDI signal is coming back immediately without any processing. This gives us a clean timing reference both in relation to the internal clock and also to the sound sent from the DX (because the sound can never come before the MIDI data).

results:
buffer 1024: MIDI is spot on, Audio is about 40 ticks early
buffer 512: MIDI is spot on, Audio is about 18 ticks early
buffer 256: MIDI is spot on, Audio is about 4 ticks early
buffer 128: MIDI is spot on, Audio is about 2 ticks late
buffer 64: MIDI is spot on, Audio is about 3 ticks late
buffer 32: MIDI is spot on, Audio is about 6 ticks late

My preliminary conclusion is that Logic overcompensates massively the larger the buffer gets. The "sweet spot" seems to be around 128 to 64 samples buffer size; I would expect a small latency inside the DX7 as well so maybe the 32 samples buffer is closer to the real situation. This could only be measured by an oscilloscope (to check how long the DX takes to react to MIDI).
MIDI in this setup was rock solid; maximum deviation for the loopback was 1 tick for all cases.

I've calculated a "recording delay" of around 917 samples for the 1024 samples buffer size. This worked well for 60, 120, 240 BPM, with higher tempos (360, 480 BPM) things started to fall apart (significant MIDI jitter up to 15 ticks, Audio early up to 100 ticks!). No idea what happens with material that has rapid runs and so on.

Since I was interested what happens with other programs I also checked with Ableton Live 10, no driver error compensation.

buffer 1024: MIDI is 6x 1/256 late, Audio is 7x 1/256 late
buffer 512: MIDI is 3x 1/256 late, Audio is 4x 1/256 late
buffer 256: MIDI is less than 2x 1/256 late, Audio is less than 3x 1/256 late
buffer 128: MIDI is 1x 1/256 late, Audio is less than 2x 1/256 late
buffer 64: MIDI is less than 1x 1/256 late, Audio is more than 1x 1/256 late
buffer 32: MIDI is less than 1x 1/256 late, Audio is slightly more than 1x 1/256 late

Preliminary conclusions: For all buffer settings in Ableton Live the audio data arrived slightly after the loopback MIDI. The timing difference between MIDI and audio stays constant. The latency for large buffer sizes is quite high; and I'm somewhat surprised that it seems to affect MIDI and audio in an equal way.
I have no idea if the delay mostly occurs on the sending or receiving side - or perhaps on both. It's somewhat difficult to make precise measurements in Ableton Live because the exact location of events isn't displayed. So pocket calculator to the rescue: What's the length of one 1/256 at tempo 120?
1/4: 500 ms
1/8: 250 ms
1/16: 125 ms
1/32: 62.5 ms
1/64: 31.25 ms
1/128: 15.625 ms
1/256: 7.8125 ms.

So the latency at a buffer size of 1024 is around 47 ms for MIDI and 55 ms for audio.

A test for driver error resulted in a 2.8 ms error. Tests with the appropriate offset didn't show much difference which doesn't come as much of a surprise to me: The error is quite small and won't make much of a difference for larger buffer sizes anyway. For smaller buffer sizes there was a slight improvement.

And last but not least: Cubase 10.5
I did not run detailed tests here, mostly because I'm not familiar enough with it. A small test with a buffer size of 1024 seemed to look similar to Ableton Live. I was somewhat irritated that the list view for the MIDI data showed me notes that were completely on spot - but the arrangement view clearly showed the notes being late... Perhaps I just overlooked some setting.

Anyway, that's all for now. I would be interested if those things could be reproduced. The overcompensation situation looks like a possible bug to me.
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Thu May 20, 2021 9:25 am

Wow thanks, really appreciate the time you put in to this gacki. I will read it tonight.
 
JakobP
Posts: 1975
Joined: Sun Aug 17, 2014 1:49 pm
Location: Stockholm

Re: midi latency at recording, not the average problem

Thu May 20, 2021 2:38 pm

Will software monitoring on or off make a difference ? I.e. shouldn't Logic compensate for both input and output latency when SM enabled, but only input latency when it's off ?
Logic Pro 10.5.1 • MBP 15" Early 2013 • OS X 10.14.6 • 2.4 GHz Intel Core i7 • 8GB RAM • 750GB SSD • Mac Mini Late 2012 • OS X 10.14.6 • 2.5 GHz Intel Core i5 • 16GB RAM • 240GB USB3 system SSD • Apple Thunderbolt Display • Focusrite Saffire 6 USB • Presonus Inspire 1394 • Roland UM-One mkII • Behringer FCB1010 • Gibson ES335 • cheap midi keyboard
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Thu May 20, 2021 9:37 pm

JakobP, i dont use software monitoring. And the problem im experiencing is where the midi is placed after the recording is made.
 
User avatar
Dewdman42
Posts: 3270
Joined: Tue Sep 09, 2014 3:01 pm
Location: Salt Lake City, UT

Re: midi latency at recording, not the average problem

Thu May 20, 2021 10:18 pm

This topic comes up every once in a while do some searching to see what has been written in the past. What I can say is that LogicPro most definitely does set the record offset of midi events as you record them, but it not be doing it at the time you would like it to, particularly because you are trying to use direct monitoring.

I don't really want to get into the weeds trying to explain the details on this, but what I can say is that in the past I spent a lot of time trying to figure it out and I remember a few observations.

  1. LogicPro places the midi notes at the point in time where it figures that you heard it, after latency of the sound. If you are using a software instrument, it knows there is some latency between when you hit the note, and going to the instrument, through plugins and out the DA converter...and it will assign the midi event to the track to line up with when it thinks you actually heard that sound relative to the click you also heard. it does NOT place the midi at the point in time where it detected that you hit the actual midi note on your keyboard. It assumes that people will naturally adjust their playing to play ahead of the beat if necessary for the latency, etc.. and will place the midi event at the point in time where it calculates that you heard the sound from that midi event.

  2. It handles external instrument plugin and direct external midi tracks in different ways for this aspect. The assumption for external midi tracks is that in those old days people would be monitoring their external keyboard through an external mixer (not in the logic Mixer)...so if you use an external midi track to direct external keyboards...then the timing will actually be more in line with when you actually struck the key. LogicPro is basically going to assume there is no latency with that external keyboard, and you're monitoring it in a latency-free external mixer. But the external instrument plugin has some latency according to whatever is happening in the the mixer PDC and the assumption is that the audio is being fed back into the LogicPro mixer...with latency, etc..so then the midi event will be registered in the track, as I said...at the point of time LogicPro calculates that you heard sound.

  3. Software monitoring vs direct monitoring has different results, which I can't remember now. I while back there were some discussions about setting up INPUT buses and different things, and handling with and without software monitoring, and honestly I can't remember what was determined, but there might be a way to play with those things in order to use direct monitoring and get a tighter midi recording, that records the midi events exactly when you play it, rather then when LogicPro thinks you heard it.
OSX 10.15 (Catalina) on OpenCore - Logic Pro 10.6.1, VePro7, Mainstage3
5,1 MacPro 3.46ghz x 12 128gb ram
 
mrcarlzone
Topic Author
Posts: 36
Joined: Thu May 06, 2021 7:13 am

Re: midi latency at recording, not the average problem

Thu May 20, 2021 11:29 pm

@Dewdman42 about number 2 on the list. My theory is that Logic records the midi when you press the key, almost no latency and no latency compensation. That means that if what the musician reacts to is delayed, which it is because of digital to audio conversation taking time, then the midi will sit late on the midi region. Is this what you are saying also? I can't quite work out if that is what you mean.
 
User avatar
Dewdman42
Posts: 3270
Joined: Tue Sep 09, 2014 3:01 pm
Location: Salt Lake City, UT

Re: midi latency at recording, not the average problem

Thu May 20, 2021 11:37 pm

LogicPro knows about not only plugins but also about your soundcard latency. That is why you hear the click exactly as the transport changes the beat number..because LogicPro knows that your sound card has latency so it makes the clock go later to match when you actually hear it. When you watch the clock, it should be matching exactly what you hear, after the metronome has gone through sound card latency.

Likewise...it knows that if you hear the sound late, due to either soundcard latency or plugin latency or both, it knows when you are hearing it and it will store the midi event into the region when you heard it, not when you played it.

This is not wrong behavior. it is a decision they made to handle it that way, which might not be what some people want, but that is intentional behavior. As I said, the assumption is that people will tend to adjust their playing so that the sound heard will be on the grid. Even a real piano has some inherent latency in the mechanisms. When people play piano, they play ahead of the grid so that the sounds will be on the grid. When peeople play an electric guitar, if the amp speaker is some distance away, there is latency between when you play it and when you hear it...and people will intuitively play ahead of the grid so that it sounds on the grid. Apple assumes that when you HEARD it is when in time you wanted it to be reproduced later. That is just a design decision.n. I don't necessarily agree with that either by the way, but that is how it is.

You might be able to find a tricky work around using external midi tracks or something to record your midi tracks without this assumption...then reassign that track later to a software instrument.
OSX 10.15 (Catalina) on OpenCore - Logic Pro 10.6.1, VePro7, Mainstage3
5,1 MacPro 3.46ghz x 12 128gb ram
 
User avatar
Dewdman42
Posts: 3270
Joined: Tue Sep 09, 2014 3:01 pm
Location: Salt Lake City, UT

Re: midi latency at recording, not the average problem

Thu May 20, 2021 11:43 pm

ps - for me this is part of why in invested in a soundcard with ultra low latency so that I basically don't have to worry about it.
OSX 10.15 (Catalina) on OpenCore - Logic Pro 10.6.1, VePro7, Mainstage3
5,1 MacPro 3.46ghz x 12 128gb ram