compile time memory leak detection

[ik]

Hello list,

There are sometimes way to "see" that you have memory leak in a code without
using any sort of profiler.
For example using new/getmem on a local variable without releasing it etc...

Does FPC know how to find this places on compile time and at least issue a
warning about it ?

Thanks,
Ido

http://ik.homelinux.org/

[Marco van de Voort]

Post by ik
There are sometimes way to "see" that you have memory leak in a code without
using any sort of profiler.
For example using new/getmem on a local variable without releasing it etc...
Does FPC know how to find this places on compile time and at least issue a
warning about it ?

Is this a suitable task for a compiler? Maybe it is a task better suited to
an external profiler.

[ik]

http://ik.homelinux.org/

Post by ik

Post by ik
There are sometimes way to "see" that you have memory leak in a code

without

Post by ik
using any sort of profiler.
For example using new/getmem on a local variable without releasing it

etc...

Post by ik
Does FPC know how to find this places on compile time and at least issue

a

Post by ik
warning about it ?

Is this a suitable task for a compiler? Maybe it is a task better suited to
an external profiler.

Well I find myself forgetting from time to time to close allocated memory on
a local variable, and a nice warning can make it easier to figure out then
running code and finding it after x amount of execution time (and using the
feature that uses that code).

So, I do not know if it's really a compiler task, but it sure can help :)

Post by ik
_______________________________________________
http://lists.freepascal.org/mailman/listinfo/fpc-devel

Ido

[Jonas Maebe]

Post by ik
There are sometimes way to "see" that you have memory leak in a code without
using any sort of profiler.
For example using new/getmem on a local variable without releasing it etc...
Does FPC know how to find this places on compile time and at least issue a
warning about it ?

No, it doesn't. In principle, the compiler can't even be sure that you
haven't replaced the default memory manager with e.g. some garbage-
collector-style memory manager similar to Boehm GC, in which case such
warnings would be wrong.


Jonas

[Nikolai Zhubr]

Post by Jonas Maebe

Post by ik
Does FPC know how to find this places on compile time and at least issue a
warning about it ?

No, it doesn't. In principle, the compiler can't even be sure that you
haven't replaced the default memory manager with e.g. some
garbage-collector-style memory manager similar to Boehm GC, in which
case such warnings would be wrong.

May I put it a bit other way?
Would it be possible/feasable to introduce some (optional) restrictions
and/or extensions to pointer-related syntax and/or pointer-related parts
of RTL so as to completely ensure safety if pointer operations?

This would probably include:
- guaranree that dynamically allocated will not be silently lost (leak)
- guaranree that dynamically allocated mem will not be disposed twice;
- guarantee that dynamically allocated mem will not be accessed beyond
the respective allocation size.

I'd guess this would require huge work and substantial modifications to
the language though. And probably, this is not possible without GC, and
I don't like GCs anyway :)

Nikolai

Post by Jonas Maebe
Jonas
_______________________________________________
http://lists.freepascal.org/mailman/listinfo/fpc-devel

[Thaddy]

Post by Nikolai Zhubr
I'd guess this would require huge work and substantial modifications
to the language though.

The compiler then has to assert all possible codepaths... Do you think
that's a viable option?

[Nikolai Zhubr]

Post by Thaddy

Post by Nikolai Zhubr
I'd guess this would require huge work and substantial modifications
to the language though.

The compiler then has to assert all possible codepaths... Do you think
that's a viable option?

I don't have that much expertise to judge really, but I'd suppose this
can't be done at compile time completely.
On the other hand, refcounting at run-time could work, theoretically.
(IIRC long strings use refcounting already)

Nikolai

[JoshyFun]

Hello Nikolai,

Tuesday, January 19, 2010, 1:38:53 PM, you wrote:

NZ> This would probably include:
NZ> - guaranree that dynamically allocated will not be silently lost (leak)
NZ> - guaranree that dynamically allocated mem will not be disposed twice;
NZ> - guarantee that dynamically allocated mem will not be accessed beyond
NZ> the respective allocation size.

That's impossible to the compiler, it is a runtime task (heaptrc):

procedure weird();
var
p: pchar;
begin
GetMem(p,10);
inc(p,random(30));
p^:='A';
end;

[Nikolai Zhubr]

Post by JoshyFun
var
p: pchar;
begin
GetMem(p,10);
inc(p,random(30));

Yes, this is among things which should probably be immediately
disallowed at compile-time (as long as one wants reasonably safe
pointers) because they are hard (or expensive) to validate later.

I usually try to not use raw pointer arithmetics and either use
new/dispose with typed pointers or constructs like GetMem(X,
SizeOf(X^)). However, compiler does not force me to do so.

Nikolai

[JoshyFun]

Hello Nikolai,

Post by JoshyFun
var
p: pchar;
begin
GetMem(p,10);
inc(p,random(30));

NZ> Yes, this is among things which should probably be immediately
NZ> disallowed at compile-time (as long as one wants reasonably safe
NZ> pointers) because they are hard (or expensive) to validate later.

NZ> I usually try to not use raw pointer arithmetics and either use
NZ> new/dispose with typed pointers or constructs like GetMem(X,
NZ> SizeOf(X^)). However, compiler does not force me to do so.

Pointers and safe code is mutually exclusive, like managed code and
fast execution, garbage collection and memory optimization, ...

In example this will work in some machines and crash in others:

GetMem(p,1024*1024*1024);
p^:=#0;

You can not detect it a compile time in any way.

[Nikolai Zhubr]

Post by JoshyFun
Pointers and safe code is mutually exclusive, like managed code and

I use pointers quite a lot. The code then works 24/7 for months under
some continuous load. (Still, I'd be even happier if compiler/RTL
double-checked my code additionally.)

Post by JoshyFun
fast execution, garbage collection and memory optimization, ...

I'm no expert in GCs, but it seems too general.

Post by JoshyFun
GetMem(p,1024*1024*1024);
p^:=#0;
You can not detect it a compile time in any way.

Sure, however this code is actually safe. (if allocation was
successfull, then execution goes on normally, otherwise an exception
will be raised on all reasonable platforms I suppose)

[Marco van de Voort]

Post by Nikolai Zhubr

Post by JoshyFun
p: pchar;
begin
GetMem(p,10);
inc(p,random(30));

Yes, this is among things which should probably be immediately
disallowed at compile-time (as long as one wants reasonably safe
pointers) because they are hard (or expensive) to validate later.

To the compiler random is just another procedure. It doesn't grasp that it
gives back random values.

FPC is not going to forbid standard pointer use, since it breaks a lot of
code.

Post by Nikolai Zhubr
I usually try to not use raw pointer arithmetics and either use
new/dispose with typed pointers or constructs like GetMem(X,
SizeOf(X^)). However, compiler does not force me to do so.

In theory, of course a dialect, or dialect switch could be implemented. It
could be even combined with an ISO Pascal mode. But the question is always
who is going to do it, and who is going to support it long term?

[Nikolai Zhubr]

Post by Marco van de Voort
FPC is not going to forbid standard pointer use, since it breaks a lot of
code.

Of course. I think it all was about new features for new code.
By the way, it reminds me MemAvail (though not exactly the same). Lots
of code used to use it, however AFAIK it is not available anymore (for a
clear reason that it is inappropriate for any modern code)

Post by Marco van de Voort

Post by Nikolai Zhubr
I usually try to not use raw pointer arithmetics and either use
new/dispose with typed pointers or constructs like GetMem(X,
SizeOf(X^)). However, compiler does not force me to do so.

In theory, of course a dialect, or dialect switch could be implemented. It
could be even combined with an ISO Pascal mode. But the question is always
who is going to do it, and who is going to support it long term?

This is understood. I was just speculating theoretically.

Nikolai

[Marco van de Voort]

Post by Nikolai Zhubr

Post by Marco van de Voort
FPC is not going to forbid standard pointer use, since it breaks a lot of
code.

Of course. I think it all was about new features for new code.
By the way, it reminds me MemAvail (though not exactly the same). Lots
of code used to use it, however AFAIK it is not available anymore (for a
clear reason that it is inappropriate for any modern code)

Totally different issue. The problem is that there was no implementation
of memavail that would suit the usage of memavail in old dos code.

That was the simple difference between single- and multiprocessing, which
was ingrained in the use. Even the simple showing available memory like
Turbo Vision apps sometimes did would lead to strange behaviour (one minute
it would look like memory was running low, and then later when an additional
block was allocated from memory, there would be plenty free memory again)

It would become even more confusing when you tried to make decisions based
on it. (like trying to allocate a big as possible block to be used as
cache). When one would let memavail return the memory available in the heap
currently allocated it was too little. If you'd let it return the memory
globally available, your risk starving other apps for memory (*), and
allocating huge chunks to former 64k apps that won't use more than a few
100ks.

There was really no solution then.

(*) not to speak of the fact that a lot of systems returned virtual memory
as free memory.

[Nikolai Zhubr]

Post by Marco van de Voort
Totally different issue. The problem is that there was no implementation
of memavail that would suit the usage of memavail in old dos code.
That was the simple difference between single- and multiprocessing, which
was ingrained in the use. Even the simple showing available memory like
Turbo Vision apps sometimes did would lead to strange behaviour (one minute
it would look like memory was running low, and then later when an additional
block was allocated from memory, there would be plenty free memory again)
It would become even more confusing when you tried to make decisions based
on it. (like trying to allocate a big as possible block to be used as
cache). When one would let memavail return the memory available in the heap
currently allocated it was too little. If you'd let it return the memory
globally available, your risk starving other apps for memory (*), and
allocating huge chunks to former 64k apps that won't use more than a few
100ks.
There was really no solution then.

Well, you are right of course. I just meant that a sole existance of
some code relying on some feature does not necessarily prove this
feature is sane enough for all times.
Standard pointer arithmetics is not inherently broken as memavail. It is
even usefull. However, a small mistake in such code also leads to
strange behaviour, and we humans tend to sometimes make mistakes...

Post by Marco van de Voort
(*) not to speak of the fact that a lot of systems returned virtual memory
as free memory.
_______________________________________________
http://lists.freepascal.org/mailman/listinfo/fpc-devel