Memory Hell

I am in what I can only describe as Logic Memory HELL.....

 

MY SYSTEM:

 

Quad 2.66 Mac Pro

9GBs RAM

Mac OS X 10.5.5

Logic Studio 8.0.2

Apogee Ensemble (audio interface)

 

I am getting crashes constantly. I seem to constantly be running out of memory on every project I'm working on now.

 

The latest crash I had was a memory crash when I tried to add another virtual instrument to my project (we'll call it Project A). It read:

 

"Logic Pro:

Not enough memory to open plug in.

(-10108)"

 

I occasionally get:

 

"Not enough memory to create clipboard/Undo.

Should never happen."

 

(paraphrasing.... though I'm certain of the SHOULD NEVER HAPPEN part. Is Apple ridiculing me?)

 

Mind you, Project A seems entirely small in terms of things I've worked on.

 

It has about 20 audio tracks and 12 instrument tracks. The only 3rd party being the latest version of Native Instruments: Battery 3.

 

Now, when I had the crash.... my Activity Monitor was reading, the following:

 

RSIZE: 1.70 GB

VSIZE: 3.97 GB

 

I've read online, that if your virtual memory exceeds 4 GBs, you will have a crash. Can someone tell me why my virtual memory on this project is so high in the first place? And why is it not using my physical memory?

 

All the troubleshooting I've been through with Apple and others:

 

1. Did a complete (extensive) hardware system test with Mac startup system DVD. No problems.

 

2. Did a memtest of all of my installed RAM.

 

3. Swapped out all pairs of RAM and still have same crashing problem on this session.

 

4. Reinstalled Logic (the Apple suggested way... deleting receipts, etc)

 

5. Created a new system user and tested... problem still exists.

 

I'm pretty certain that this is not a project specific problem, as I'm having it on several different projects. The only thing they seem to have in common is the usage of at least one third party instrument. My other projects all seem to crash around 2.3GBs of physical memory used and again 3.9 GBs of virtual memory..... It seems that virtual memory might be the problem?

 

so just now... I decided to take this EXACT project and open it up on my Dual 2.5 G5 (OS X 10.4.11) to see what happens. Switched all my cables and audio interface and hard drives. Guess what!! No crash! I can load at LEAST 5 more instruments - and while it is sluggish, NO CRASHES! NO MEMORY problems!

 

In fact, the Activity Monitor reads as follows on this computer:

 

Real: 1.72 GB

Virtual Memory: 3.11 GB

 

Significantly less Virtual memory being used and this is after I loaded up 5 more EXS 24 instruments that made the other system crash.

 

What does all of this mean?? Is it Intel? Is it Leopard? Is it some hardware issue that fails to be detected by a hardware test??

 

Some Apple tech is supposed to call ME in the morning... I will at least be able to report this new finding.

 

It's a sad day when my older computer is more stable than the new one. I thought I was safe upgrading now.... how long have these Intels been out??

 

Oh... and btw, I'm having the same TERRIBLY dramatic slowdown of the GUI as I work. It's not cool. I'm about to start a big project and I don't know if I should stop using Logic and switch to something else!! It just seemed so right with Apple and all....

 

Lucian

Sounds like a hard drive problem - (I'm sure I don't need to ask but are you backed-up?). Does the same happen in other cpu hungry programs?

What does it have to do with my hard drive? I realize that virtual memory is stored on the hard drive, right? Still don't really understand what the problem would be then....

 

btw I repaired disk permissions and all that. I have plenty of drive space free on all of my drives.

virtual memory is stored on the hard drive

That's my understanding.

TERRIBLY dramatic slowdown of the GUI

and the fact that it works on another computer could be symptomatic of potential failure - and you say you have plenty of free space left on your system drive (at least 10% if not more). But others may have better ideas or similar experiences. I only dream about having a more powerful system.

Hi revolucian,

 

Sorry to hear about your problems. Are you able to recreate the issue in a new project with the same plugins and such? If not then there is something in common with the ones that have the issue that is causing it to happen.

 

Also, you might want to export the project as an XML from the G5 and import the XML into Logic on the Mac Pro and see if that helps.

 

Were all of the projects where it happens created in LP8 on this system or on your G5 in LP8, 7?

 

If you can reproduce the issue with new projects on the Mac Pro then it very well could be a Leopard problem or an issue with the 3rd party plugins running on the Mac Pro.

Well I am definitely having the issue on newly created files that include KontaktPlayer and Battery 3.... though the memory does go up to:

 

RSIZE: 2.3

VSIZE: 3.97

(approx)

 

before it crashes.

 

I've been reading a lot of mixed information. Some things saying that Logic can handle up to 4 GBs of memory... others saying 2

 

To answer your questions, this specific Project A was created on the G5. I then opened it on the Mac Pro. All of the same plug-ins are installed on both computers.

 

Either way, it actually seems to be the Virtual memory maxing out that is causing these crashes.

 

I will try that XML export on the G5...

It sounds to me like you have already isolated it to the NI plugins on the Mac Pro. Would not focus on the old projects when it happens on new ones too. I'm assuming you've run the NI Update Manager and installed all the available updates for your installed plugins? Might want to email NI and let them know the issue as well and see if they can help.

 

Also, did you migrate the software to this system from your G5 or was it all installed from installers?

 

Cheers

I think I'm determining it has something to do with how Logic deals with ALL 3rd party plugins.

 

I have Soundtoys as well. I had about 10 instances of Echoboy on this session and Activity Monitor was reporting that fx plugin using about 500 MB of RAM? That's crazy. I emailed Soundtoys, though.

 

And yes, all Native Instrument stuff is up to date.

 

It seems that this crash always happens around 4GB of virtual RAM usage in Leopard. And for some reason Leopard is using a lot more RAM than my G5 running 10.4.11

I've read online, that if your virtual memory exceeds 4 GBs, you will have a crash.

 

I don't know the limitations for 3rd party plugs and VM but as I write this my system is operating on

 

Inactive - 2.09 GB

Used - 2.57 GB

Free - 1.93 GB

VM size - 6.78 GB

 

It's all EXS and built in plugs.

 

Please post any info you get from Apple.

I'm able to run any NI plugin from Komplete 5 on my Mac Pro Quad in LP 8.0.2 on 10.5.5 and they run fine. No crashes. There a loads of people running 3rd party plugs in Leopard w/out issue. Something specific to your system is causing this.

ok..... about how many instances of these plugins can you load? like Battery 3... or something like that.... before you have memory problems? Does anyone have memory problems with 9GBs of memory on a Quad Mac Pro?? It seems to be a relatively small number of virtual instruemnts I add that maxes out my system.

 

I've also noticed that Soundtoys native effects take up large amounts of Virtual Ram.... I load about 40 instances of echoboy or filterfreak and logic crashes.... with NO virtual instruments!

 

Something is wrong right?? Maybe just for me.

revolucian,

 

it is not just you, it is what you are attempting to do. people not having these issues are doing different things. i know a fair bit about this issue having been stuck with it on a big project for the last few months. ironically it might be because you have too much RAM installed. here's the skinny...

 

logic 8 uses a special feature called mmap to map out samples from the exs instrument to memory space outside of logics own 32bit 3.5 GB limit. in order to use this feature you need to have at least 5Gb RAM installed and virtual memory in the exs prefs turned on. it allows for at least 3.5 Gb of samples to be loaded per exs instance.

 

it has no effect on 3rd party plugs however.

 

the feature 'robs' logic of around 512 Mb of internal RAM in order to carry out the remapping, which reduces further logics own rather low (by todays standards) ceiling.

 

when you load up a 3rd party plug, it reserves memory for its samples and for its UI which can be as much as 200 Mb. that's really a lot and loading in samples will suck even more memory. add to that your compressors, reverbs and their IRs, loops (which have to be held in RAM) and programming and then you find logic starts to choke.

 

the problem with mmap, is that while it means you can load exs instruments of virtually unlimited size, it reduces the number of 3rd party plugs you can run. so ironically, if you were to reduce you RAM down to under 5 GB, mmap would not operate and actually you could probably run all your 3rd parties. at least, i can run projects without issue in 7 that are causing problems in 8.

 

not only that, but even you do manage to run the projects, you will either run out of memory as you program, or not be able to switch songs (you will have to reboot logic). so it is fair to say that the mmap facility has some room for improvement.

 

however there is a good workaround solution which i am going to be going for on my next projects:

 

load your 3rd party plugs into au lab or plogue bidule. with plogue bidule, you will be able to load up your instruments and then pipe them into an aux channel strip on logic. then you won't run foul of the memory limits, you will still get signal flow in logic ITB, and you can enjoy all that RAM you have installed. there are other alternatives too: RAX vbus and sbus for apogee gear, and a few others i have forgotten about already....

Hi All

 

I have recently bought a new Mac Pro 8 Core system which has 10GB of Ram installed. Everything has been great, untill now when i start getting these memory errors also.

 

I use mainly all software synths, Komplete 5 with the Kore 2 Controller along with all the others, Rob Pappen, Re FX etc etc.

 

After reading this post i noticed about the Virtual memory problem, when i open this particular project, its already using 3.3GB of VM. If i do a simple save (before even playing through or recording any midi or loading any more synths) the VM jumps up to 3.6 another save pushes it to 3.8 and then 1 more save pushes it to 3.99GB where it crashes.

 

Is there not a way to increase the VM or is logic only capable of addressing up to 4GB? Why doesnt it use Real memory first? As when it crashes it says its only addressing 2.17GB of 'Real Memory'?

 

 

Bite my balls Apple..

I'm sure you have already considered this, but make sure the ram is in the right slot and that you are using either 2, 4, or 8 slots.

 

I just bought a macpro and you guys are scaring me. I still have the stock 2 gigs, maybe ill stick to 4 and keep this mmap ccrap off...

Don't worry about VM. It is paged out and other stuff is paged in. You should worry about the page in/page out numbers, that would indicate that the instruments use a lot of memory if those numbers are constantly updated at a fast pace. You should even hear it in forms of constant hard disk churning or something similar.

revo...can you post a quick screen shot of your RAM from your System Profile (like this?)

 

I have heard enough people say they have a lot of issues when using only 6 RAM slots. Have you tried using JUST your new RAM with only 2 in Riser A and 2 in Riser B?

this problem has absolutely nothing to do with your physical RAM or in what sots they are in or swapping them out.

 

it is a limitation with 32bit logic.

 

i had a friend who does some developing alter the amount of RAM needed for mmap in my plist file from 512 to 256 and projects that were struggling (sluggish etc) suddenly performed a lot better. maybe a fix for this will turn up later with logic.

 

it is only going to be a problem if you are loading really a lot of samples. but samples themselves are not the only issue.

 

when you instantiate a plug-in in logic it reserves memory - internal RAM of a specific size. that occupies a block of memory so that even though you appear to have lots of memory available, it is not in continuous blocks of sufficient size. a bit like someone parking across 2 bays. there might be enough room for 2 more cars but they can't get in because one car is straddling 2 of the bays.

 

the solution if you start running into memory problems is to use an au host. au lab is free and works brilliantly but i don't think it is rewire compatible. plogue bidule is and you can load heaps into that and rewire back into logic. its the only way of benefitting from more than 4 Gb of RAM.

HI guys and thanks all for replying. Can someone clarify for me / us what the exact limit of memory is, what the ceiling is? I spoke to Apple the other day, and he said its 2GB, i thought it was 4GB? and should the virtual memory go way beyond 3.99GB where its crashing? Or is this the 4GB limit being reached? Im so confused now! thought logic could address 4GB which is why i am confused that its crashing when my REal Memory says 2.17GB

 

Also Stevenon, i ended up downloading that Bidule program. Great piece of work. This might be totally impossible, but do you know if there is anything you can put into logics environment so that you can rewire Bidule and have that audio quantized? I do a lot of 4 to the floor dance stuff so quiet often i require hard quantization. Im going to start a new post for this so this post is kept on topic.

Go to Preferences >Audio> General

Then

Turn off Sample Accurate Automation

This isnt really necessary and it eats up CPU load especially on parts where there is a lot going on like running a lot of software instruments at a given time.

logic is a 32 bit application. it can address just shy of 4 Gb memory in total, but some 500 Mb of that is required for the actual program (maybe not quite that but lets keep things simple).

 

your man at apple didn't really know what he is talking about from the sounds of things.

 

the figure you want to keep in mind when looking at is the 'virtual memory' figure in activity monitor, not to be confused with 'virtual memory' in the exs preferences. in logic 8, the most you will be able to get logic up to is 3.5 Gb before very nasty things will happen. 8 will stay afloat longer than 7 with high memory usage, but you will be able to load more things into 7 as it is a smaller program and will not use mmap.

 

this whole area is pretty confusing, and the devs i have talked about this issue didn't sound like they were without confusion themselves.

 

the bottom line is: if you want to exploit modern computers fully, logic alone will not have sufficient internal memory to host everything they can pull. if you have problems with sluggishness, instability, or out of memory errors, you should offload your 3rd party stuff into an AU host such as bidule. this doesn't just go for people running large sample libraries. even people just working with audio and effects can run into the memory problem.

Clean out your registry. Get a program called Onyx. http://www.macupdate.com/info.php/id/11582

This is a great program for removing all the junk in your computer that you cannot see or pay attention to. It will look like it is doing some scary stuff, deleting caches, repairing disk permissions. You want to do these things. I use this program all the time and it is the reason my mac runs as good as the day I got it. (!!!Important!!!) When you are deleting the caches pay attention to which ones are checked off. There is an option to delete all audio au components obviously not what you want to do. Uncheck this one. AND! there is a a cache for other audio stuff you will need. Just make sure these are not checked and you are golden. I also make sure I delete all of my unwanted audio files in my Logic sessions. This is very important to make sure your sessions play flawlessly if you are doing multiple takes for audio. Open logic with all of your sessions and repeat this process. Click on media in the upper right hand corner. Open your bin. Click on Edit and choose select unused. You will be selecting your unused audio files in the track that arent even playing!!! They eat up system resources and they are making your tracks overweight in a way. I like to refer to this as selecting the excess fat on your project to be lypoed. Go to Audio File and click Delete files. It will scare you with a window saying are you sure you want to delete these files they will be gone forever. Do not be scared by this you want this. After deleting these files play back your track you will indefinitely see that nothing is missing and everything is there. Somehow I think that this actually improves sound quality or performance but it could be my imagination. It definitely improves performance and you delete unwanted files in your HD!!!! This will fix your issue more than both of these options put together, however I reccomend doing all of these procedures. Convert all of your software instrument tracks to Audio tracks. If your system cannot perform all of your AU`s and Midi information do away with it. It is rubbish!!! Audio plays much easier than a software instrument track with midi. When I used to have pro tools with an old pc I would do this so my projects wouldnt crash. The same works very well in logic. I used to bus an output from my software instrument tracks then take that same bus aux to input into an audio track then record. i am afraid that this might reduce sound quality so I found a better option that doesnt potentially reduce sound quality and makes your session play effortlessly forever!!!!!! Select one of your software or external midi tracks so all of your regions are selected. If you have seperate regions make sure have them all highlighted then click on Region scroll down to merge and select merge regions. You want to do this so all your midi will be one region. Then hold control and click on your region select export as audio file. I send them to the desktop to be deleted later because they are automatically saved into your sessions audio file folder. Add a new Audio track set the bar to the appropriate measure if you will. Then hold Command>Shift>I to import an audio file. Go to your desktop and you will see your saved file. Click on it and shazaam you have an audio file to replace your midi file with the exact same audio quality. Continue this process on all of your tracks. You will indefinitely see that your sessions will never crash again.

Edited January 13, 2009 by Dedmondson

Good luck!!!

Onyx looks cool..hadn't heard of it. I use Cocktail for pretty much the same thing...

 

http://www.maintain.se/cocktail/index.php

 

 

-

Hey folks.... so my latest work-around for all of this is to simply keep activity monitor open and watch as I approach 4.0 GBs of VM where it ALWAYS crashes.

 

I basically had to cease using SoundToys plug-ins because they seem to take up an obnoxious amount of Virtual Memory.... They also put a sort of blockade on your Real Memory usage that makes all other plugins go to Virtual Memory instead of Real Memory. That's the only way to describe what I've experienced. This 'blockade' that I'm speaking of is similar to the terribleness that makes me have to QUIT LOGIC every time I switch between projects, because the memory is not released. WTF Apple?

 

This is not any sort of memory problem.... the risers or memory amounts or pairings or whatnot don't make a difference. This is about how Logic handles memory in Leopard on an Intel(at least in my case) system.

 

It's really unfortunate because I have projects that run out of memory as they're opening on my Intel Quad that open FINE on my Dual 2.5 G5. That's stupid.

 

Hopefully, Apple gets their act together. I have a feeling we are being neglected. Logic 8 has some great enhancements but there are some obvious bugs that are killing me.

 

Perhaps Snow Leopard will be the solution we need.

Hey folks.... so my latest work-around for all of this is to simply keep activity monitor open and watch as I approach 4.0 GBs of VM where it ALWAYS crashes.

 

I basically had to cease using SoundToys plug-ins because they seem to take up an obnoxious amount of Virtual Memory.... They also put a sort of blockade on your Real Memory usage that makes all other plugins go to Virtual Memory instead of Real Memory. That's the only way to describe what I've experienced. This 'blockade' that I'm speaking of is similar to the terribleness that makes me have to QUIT LOGIC every time I switch between projects, because the memory is not released. WTF Apple?

 

This is not any sort of memory problem.... the risers or memory amounts or pairings or whatnot don't make a difference. This is about how Logic handles memory in Leopard on an Intel(at least in my case) system.

 

It's really unfortunate because I have projects that run out of memory as they're opening on my Intel Quad that open FINE on my Dual 2.5 G5. That's stupid.

 

Hopefully, Apple gets their act together. I have a feeling we are being neglected. Logic 8 has some great enhancements but there are some obvious bugs that are killing me.

 

Perhaps Snow Leopard will be the solution we need.

 

The only way to allocate 'wired memory or what you call real is to make specific calls to use it via the kernel. Any applications or plug-ins use malloc and its top level APIs to allocate memory and it always ends up virtual memory. So what you describe does not happen unless you have a very buggy device driver installed or something similar.

 

It's very little an OS could do about allocating memory than to just try to support any plug-ins that need it. When the application quits any memory allocated via malloc will be freed as part of the app process instance. If it's not released properly during the application time it is a memory leak and the VM size will of course increase as no pages are swapped out while more pages are needed. Again, if a plug-in has memory leaks, little the OS could do as it can't go in an orchestrate releases of memory concerning code it knows nothing about concerning the execution.

 

There's always garbage collection in Leopard that would help to some degree, but there are also some know-how that a developer needs to know about using garbage collection, like not hanging on to objects for the lifetime of the app instance. It also assumes that the code will make use of Objective-C 2.0 runtime. A lot of plug-ins are really C/C++ based due to cross-platform and generic audio algorithms implemented. Few plug-ins are even using Cocoa today for MacOSX.

 

My best advice with these kinds of cases is:

a) contact the plug-in manufacturer about the problem. There might even be a new version that fixed the problem, or you found something the developer was not aware of and keen to fix.

b) Max out your memory on your system, memory is so cheap nowadays so it's a matter of taking a trip to a computer store, getting stick, and stuffing them inside the computer. This will of course not fix memory leaks, but it will minimize the amount of virtual page swapping which is really the killer concerning performance if the OS needs to go to the file system for paging in and out memory pages.

I seriously could kill someone at APPLE right now!!!!!!

 

Logic just FROZE on a SAVE! After 30 minutes of work!!! first I got the message "Logic was unable to save...."

 

This is all because of this @%#$$%@#$$$ memory problem!@@#@#! I looked and sure enough, Logic surpassed it's 4.0 GB of Virtual Memory limit.

 

WHY THE HELL CAN LOGIC NOT KNOW WHEN YOU ARE APPROACHING THE LIMIT OF YOUR MEMORY USAGE!@@#!?!@#?!@#$%@$T^?

 

CRASHING ON SAVES IS NOT ACCEPTABLE

 

Just had to get that out.

CRASHING ON SAVES IS NOT ACCEPTABLE

 

When is crashing acceptable?

touche

After using (programming) computers since 1977 I just don't trust them, ever. Save early, save often.

Note Kingston says this:

 

"Memory must be installed in pairs. Apple recommends four or eight identical DIMMs for best performance. Kingston offers memory in kits of two identical DIMMs."

 

But that's probably just for optimal speed -- not a matter of working or not working.