Is it possible to detect when a program is using, say, 90% of the memory that is available, so that I can take steps to reduce usage?  I've got a method that would like to use probably more memory than is available, but which can (if it could tell) save stuff to files to free memory it isn't using at the moment.  But when I looked in http://dlang.org/phobos/core_memory.html all the methods seemed to have to do with garbage collection, which isn't quite the same thing.  I don't want the stuff to be garbage collected until after I've saved it's current state, but to know when to do that, I need to know how much memory is currently free.

I suppose that I could set an arbitrary limit on the number of various things I allow before compacting, but that's a bit ... arbitrary.  And doesn't adapt to different machines without recompiling.

Charles Hixson:

> Is it possible to detect when a program is using, say, 90% of the memory that is available, so that I can take steps to reduce usage?

With computers that have virtual memory this is not so easy to do. I think you have to use operating system-specific code to ask the OS about the physical available memory, about the virtual memory used, the rate of virtual memory swapping, and use those three values in some way.

Bye,
bearophile

On Thu, 23 Aug 2012 18:37:59 -0500, bearophile <bearophileHUGS@lycos.com> wrote:

> Charles Hixson:
>
>> Is it possible to detect when a program is using, say, 90% of the memory that is available, so that I can take steps to reduce usage?
>
> With computers that have virtual memory this is not so easy to do. I think you have to use operating system-specific code to ask the OS about the physical available memory, about the virtual memory used, the rate of virtual memory swapping, and use those three values in some way.
>
> Bye,
> bearophile

On linux this is not so difficult to do.

Those values are generally in /proc, and it seems to be portable across pretty much every distro with a relatively recent kernel.

I have an extremely half-assed bit of code that prints the load average and the totaly % of mem used to my tmux session.
It gives the exact same values that are seen in top, or htop.(without the overhead of parsing their output, cause that takes ~500ms, way too slow.)
It would make a decent starting point at least.

I would imagine that you wouldn't even need to know the % of memory YOUR program is using, just the general percentage overall.

After all, no matter who is eating all the memory, shit's about to hit the fan if *someone* doesn't free some memory.
-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On 24-Aug-12 07:03, 1100110 wrote:
>
> On linux this is not so difficult to do.
>
> Those values are generally in /proc, and it seems to be portable across
> pretty much every distro with a relatively recent kernel.
>
> I have an extremely half-assed bit of code that prints the load average
> and the totaly % of mem used to my tmux session.
> It gives the exact same values that are seen in top, or htop.(without
> the overhead of parsing their output, cause that takes ~500ms, way too
> slow.)

If parsing takes 500ms then something is seriously wrong. What is size of the input to parse and the machine specs?


-- 
Olshansky Dmitry

On Fri, 24 Aug 2012 15:42:21 -0500, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:

> On 24-Aug-12 07:03, 1100110 wrote:
>>
>> On linux this is not so difficult to do.
>>
>> Those values are generally in /proc, and it seems to be portable across
>> pretty much every distro with a relatively recent kernel.
>>
>> I have an extremely half-assed bit of code that prints the load average
>> and the totaly % of mem used to my tmux session.
>> It gives the exact same values that are seen in top, or htop.(without
>> the overhead of parsing their output, cause that takes ~500ms, way too
>> slow.)
>
> If parsing takes 500ms then something is seriously wrong. What is size of the input to parse and the machine specs?
>
>
No, you misunderstand.

Parsing the output of the `top` command takes ~500ms.

The snippet of code that reads /proc directly takes an order of magnitude less time.

I assume that top reads several times, and also reads the percentages of each process that is running, while I only
read the load average and total memory consumption.

The vast majority of the time spent parsing `top` was spent waiting for it to initialize, and print the values to be read.

TL;DR
Parsing was probably the wrong word to use.

Reading from /proc directly will take an order of magnitude less time.


-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On 25-Aug-12 00:52, 1100110 wrote:
> On Fri, 24 Aug 2012 15:42:21 -0500, Dmitry Olshansky
> <dmitry.olsh@gmail.com> wrote:
>
>> On 24-Aug-12 07:03, 1100110 wrote:
>>>
>>> On linux this is not so difficult to do.
>>>
>>> Those values are generally in /proc, and it seems to be portable across
>>> pretty much every distro with a relatively recent kernel.
>>>
>>> I have an extremely half-assed bit of code that prints the load average
>>> and the totaly % of mem used to my tmux session.
>>> It gives the exact same values that are seen in top, or htop.(without
>>> the overhead of parsing their output, cause that takes ~500ms, way too
>>> slow.)
>>
>> If parsing takes 500ms then something is seriously wrong. What is size
>> of the input to parse and the machine specs?
>>
>>
> No, you misunderstand.
>
> Parsing the output of the `top` command takes ~500ms.
>
Then it's not parsing but the it's time to spawn process that does a fuckton of syscalls and read its output through the pipe :)
I just cringed at:
"the overhead of parsing their output, cause that takes ~500ms"

> The snippet of code that reads /proc directly takes an order of
> magnitude less time.
>
> I assume that top reads several times, and also reads the percentages of
> each process that is running, while I only
> read the load average and total memory consumption.
>
> The vast majority of the time spent parsing `top` was spent waiting for
> it to initialize, and print the values to be read.
>
> TL;DR
> Parsing was probably the wrong word to use.
>

Yes, sorry for being nit-picky.

> Reading from /proc directly will take an order of magnitude less time.


And yes.

-- 
Olshansky Dmitry

On Fri, 24 Aug 2012 16:05:28 -0500, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:

> On 25-Aug-12 00:52, 1100110 wrote:
>> On Fri, 24 Aug 2012 15:42:21 -0500, Dmitry Olshansky
>> <dmitry.olsh@gmail.com> wrote:
>>
>>> On 24-Aug-12 07:03, 1100110 wrote:
>>>>
>>>> On linux this is not so difficult to do.
>>>>
>>>> Those values are generally in /proc, and it seems to be portable across
>>>> pretty much every distro with a relatively recent kernel.
>>>>
>>>> I have an extremely half-assed bit of code that prints the load average
>>>> and the totaly % of mem used to my tmux session.
>>>> It gives the exact same values that are seen in top, or htop.(without
>>>> the overhead of parsing their output, cause that takes ~500ms, way too
>>>> slow.)
>>>
>>> If parsing takes 500ms then something is seriously wrong. What is size
>>> of the input to parse and the machine specs?
>>>
>>>
>> No, you misunderstand.
>>
>> Parsing the output of the `top` command takes ~500ms.
>>
> Then it's not parsing but the it's time to spawn process that does a fuckton of syscalls and read its output through the pipe :)
> I just cringed at:
> "the overhead of parsing their output, cause that takes ~500ms"
>
>> The snippet of code that reads /proc directly takes an order of
>> magnitude less time.
>>
>> I assume that top reads several times, and also reads the percentages of
>> each process that is running, while I only
>> read the load average and total memory consumption.
>>
>> The vast majority of the time spent parsing `top` was spent waiting for
>> it to initialize, and print the values to be read.
>>
>> TL;DR
>> Parsing was probably the wrong word to use.
>>
>
> Yes, sorry for being nit-picky.
>
>> Reading from /proc directly will take an order of magnitude less time.
>
>
> And yes.
>
    size_t memTotal, memUsed;
    Stream memInfo  = new BufferedFile("/proc/meminfo");
    auto t          = memInfo.readLine();
    memTotal        = to!size_t(strip(t[9..$-3]));
    memUsed         = memTotal;

    auto f  = memInfo.readLine();
    memUsed -= to!size_t(strip(f[8..$-3]));

    auto c = memInfo.readLine();
    memUsed -= to!size_t(strip(c[8..$-3]));

    auto b = memInfo.readLine();
    memUsed -= to!size_t(strip(b[8..$-3]));

    memInfo.close();

Fair warning, I wrote this over the course of 5 minutes by directly translating some public domain code.
It works, so I've never bothered to clean it up.

But that'll give you the total memory used, and the total amount of memory.
-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/