[fpc-devel] Memory leak on BeginThread() when RTL compiled with -O2

[NetSpirit via fpc-devel]

Answer for my previous message
(https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).

Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with
"-O2" or higher.
To eliminate this bug add "-O-" or "-OoNOREGVAR" to command-line when
compile this unit. This disables optimizations when enabled in 'fpc.cfg',
e.g:

%FPC_COMMAND% -O- -Mobjfpc win32\sysinitpas.pp

or

%FPC_COMMAND% -OoNOREGVAR -Mobjfpc win32\sysinitpas.pp




_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lis

[J. Gareth Moreton via fpc-devel]

Hmmm, I've seen regvar problems before.  Can you provide the assembly
dumps of that file under -O2 and -O2 -OoNOREGVAR respectively? It might
reveal something.

Gareth aka. Kit

On 20/05/2021 09:07, NetSpirit via fpc-devel wrote:
> Answer for my previous message
> (https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).
>
> Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with
> "-O2" or higher.
> To eliminate this bug add "-O-" or "-OoNOREGVAR" to command-line when
> compile this unit. This disables optimizations when enabled in 'fpc.cfg',
> e.g:
>
> %FPC_COMMAND% -O- -Mobjfpc win32\sysinitpas.pp
>
> or
>
> %FPC_COMMAND% -OoNOREGVAR -Mobjfpc win32\sysinitpas.pp
>
>
>
>
> _______________________________________________
> fpc-devel maillist - fpc-***@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
>

--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/ma

[NetSpirit via fpc-devel]

You can simply compile this file by yourself using your existing FPC
installation.
1) Get 'rtl\win32\sysinitpas.pp' and 'rtl\win32\sysinit.inc' from
repository or sources archive.
2) Create build script ('compile.bat'):

REM Change this path's to your FPC installation, e.g.:
set FPC_DIR=c:\FPC
set FPC_BINARY=%FPC_DIR%\bin\i386-win32\ppc386.exe
set UNITS_DIR=%FPC_DIR%\units\i386-win32\*

"%FPC_BINARY%" -Fu"%UNITS_DIR%" -O2 -Mobjfpc "sysinitpas.pp"

=======
Here is test project that show bug: http://dl.free.fr/ckmK7KVPX
Test project with full environment (compiler and minimal RTL):
http://dl.free.fr/nHDbSOzOQ



> Hmmm, I've seen regvar problems before.  Can you provide the assembly
> dumps of that file under -O2 and -O2 -OoNOREGVAR respectively? It might
> reveal something.
>
> Gareth aka. Kit
>
> On 20/05/2021 09:07, NetSpirit via fpc-devel wrote:
>> Answer for my previous message
>>
(https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).
>>
>> Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with
>> "-O2" or higher.
>> To eliminate this bug add "-O-" or
"-OoNOREGVAR" to command-line when
>> compile this unit. This disables optimizations when enabled in 'fpc.cfg',
>> e.g:
>>
>> %FPC_COMMAND% -O- -Mobjfpc win32\sysinitpas.pp
>>
>> or
>>
>> %FPC_COMMAND% -OoNOREGVAR -Mobjfpc win32\sysinitpas.pp
>>


_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepasca

[Yuriy Sydorov via fpc-devel]

Hello,

Your test program runs fine when compiled with FPC 3.2.0 using the official RTL compiled with -O2. No memory leak.
I suppose the issue is caused by your modifications to RTL. -O2 (regvars) just triggers the problem in the modified RTL
code which is hidden when regvars are disabled.
You need to debug the thread code in your modified RTL by yourself and find out what exactly causes the memory leak and why.
If there is indeed a bug in the regvars implementation you need to create a simple test program which triggers the bug.
This is a common practice for such issues. Only such way it can be fixed.

Yuriy Sydorov,
***@cp-lab.com

On 20.05.2021 11:07, NetSpirit via fpc-devel wrote:
> Answer for my previous message
> (https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).
>
> Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with
> "-O2" or higher.
> To eliminate this bug add "-O-" or "-OoNOREGVAR" to command-line when
> compile this unit. This disables optimizations when enabled in 'fpc.cfg',
> e.g:
>
> %FPC_COMMAND% -O- -Mobjfpc win32\sysinitpas.pp
>
> or
>
> %FPC_COMMAND% -OoNOREGVAR -Mobjfpc win32\sysinitpas.pp
>
>
>
>
> _______________________________________________
> fpc-devel maillist - fpc-***@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
>
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.f

[NetSpirit via fpc-devel]

RTL not modified. Just recompilation of 'sysinitpas.pp' unit with some
options causes problem.

Here is another test project that uses your current official FPC
installation with unmodified RTL:
http://dl.free.fr/ePtkQwgJJ

Test project contains source of 'sysinitpas.pp' (and required
'sysinit.inc', 'systlsdir.inc').
Used version 3.2.2 RC1. If you have another - get sources for your version.

Script 'compile.bat' compiles 'sysinitpas.pp' and then - test project
'test.pp'.
See comments inside.

'test_bad.exe' shows growing memory consumption.


> Hello,
>
> Your test program runs fine when compiled with FPC 3.2.0 using the
official RTL compiled with -O2. No memory leak.
> I suppose the issue is caused by your modifications to RTL. -O2
(regvars) just triggers the problem in the modified RTL code which is
hidden when regvars are disabled.

_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.free

[Yuriy Sydorov via fpc-devel]

Hello,

I've take a look at you RTL compile script and found the mess with the compiler options.

When compiling RTL you should disable reading of options from fpc.cfg using the -n switch.
When compiling the system unit use the -Us switch.
Additionally specify only CPU/Target and optimization options -O<x>.
No other options are required for Windows RTL.

When RTL is properly compiled, even with -O2, your test program works without memory leaks.

Yuriy Sydorov,
***@cp-lab.com

On 20.05.2021 15:43, NetSpirit via fpc-devel wrote:
> RTL not modified. Just recompilation of 'sysinitpas.pp' unit with some
> options causes problem.
>
> Here is another test project that uses your current official FPC
> installation with unmodified RTL:
> http://dl.free.fr/ePtkQwgJJ
>
> Test project contains source of 'sysinitpas.pp' (and required
> 'sysinit.inc', 'systlsdir.inc').
> Used version 3.2.2 RC1. If you have another - get sources for your version.
>
> Script 'compile.bat' compiles 'sysinitpas.pp' and then - test project
> 'test.pp'.
> See comments inside.
>
> 'test_bad.exe' shows growing memory consumption.
>
>
>> Hello,
>>
>> Your test program runs fine when compiled with FPC 3.2.0 using the
> official RTL compiled with -O2. No memory leak.
>> I suppose the issue is caused by your modifications to RTL. -O2
> (regvars) just triggers the problem in the modified RTL code which is
> hidden when regvars are disabled.
>
> _______________________________________________
> fpc-devel maillist - fpc-***@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
>
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-d

[NetSpirit via fpc-devel]

Thank you, i will try this.

> I've take a look at you RTL compile script and found the mess with the
compiler options.
>
> When compiling RTL you should disable reading of options from fpc.cfg
using the -n switch.
> When compiling the system unit use the -Us switch.
> Additionally specify only CPU/Target and optimization options -O.
> No other options are required for Windows RTL.
>
> When RTL is properly compiled, even with -O2, your test program works
without memory leaks.
>
> Yuriy Sydorov,
> ***@cp-lab.com


_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-deve

[NetSpirit via fpc-devel]

Unfortunately I tried it and this doesn't work.
Problem only with 'sysinitpas.pp' unit.
He is not depends on other RTL units, but linked into final executable by
compiler.
I suppose, compiler itself depends on internal layout of this unit.
And a solution is to compile 'sysinitpas.pp' with the same options as
'ppc386.exe' was compiled, not RTL.

See my previuos demo: http://dl.free.fr/ePtkQwgJJ

> When compiling RTL you should disable reading of options from fpc.cfg
using the -n switch.
> When compiling the system unit use the -Us switch.
> Additionally specify only CPU/Target and optimization options -O. > No
other options are required for Windows RTL.
>
> When RTL is properly compiled, even with -O2, your test program works
without memory leaks.
>
> Yuriy Sydorov,
> ***@cp-lab.com
>


_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://list

[Yuriy Sydorov via fpc-devel]

Hello,

You have not read closely what I wrote before:

"When compiling RTL you should disable reading of options from fpc.cfg using the -n switch.
When compiling the system unit use the -Us switch.
Additionally specify only CPU/Target and optimization options -O<x>.
No other options are required for Windows RTL."

sysinitpas.pp is the part of the RTL. So in your example it is needed to be compiled as:

"%FPC_BINARY%" -Fu"%UNITS_DIR%" -FU"%SCRIPT_DIR%units" -n -O2 "sysinitpas.pp"

-n ensures the default state of the compiler settings. This is the intended state for compiling the RTL.

RTL must be compiled in the FPC mode. No other modes must be specified in the command line or fpc.cfg. You can't first
pass the -Mdelphi switch and then override it with -Mfpc. -Mdelphi implicitly enables lot of other compiler options
which are not reverted by -Mfpc.


Yuriy Sydorov,
***@cp-lab.com

On 24.05.2021 12:00, NetSpirit via fpc-devel wrote:
> Unfortunately I tried it and this doesn't work.
> Problem only with 'sysinitpas.pp' unit.
> He is not depends on other RTL units, but linked into final executable by
> compiler.
> I suppose, compiler itself depends on internal layout of this unit.
> And a solution is to compile 'sysinitpas.pp' with the same options as
> 'ppc386.exe' was compiled, not RTL.
>
> See my previuos demo: http://dl.free.fr/ePtkQwgJJ
>
>> When compiling RTL you should disable reading of options from fpc.cfg
> using the -n switch.
>> When compiling the system unit use the -Us switch.
>> Additionally specify only CPU/Target and optimization options -O. > No
> other options are required for Windows RTL.
>>
>> When RTL is properly compiled, even with -O2, your test program works
> without memory leaks.
>>
>> Yuriy Sydorov,
>> ***@cp-lab.com
>>
>
>
> _______________________________________________
> fpc-devel maillist - fpc-***@lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
>
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/li

[NetSpirit via fpc-devel]

> Hello,
>
> You have not read closely what I wrote before:

Sorry, earlier I try your hint - and result was bad.
Perhaps I did something wrong.
But I repeat it now - and it is working. Thank you again.

Finally my working build script look like this:


set UNITS_PATH=.\packages\*
set UNITS_PATH=.\objpas;.\objpas\*;.\inc;.\win;%UNITS_PATH%;.\win32 set
INC_PATH=.\inc;.\objpas;.\objpas\*;.\win;.\win\wininc;.\i386;.\win32 set
OUTPUT_PATH=..\units32a

set FPC_COMMAND=..\ppc386.exe -n -O3 -Twin32 -FU%OUTPUT_PATH%
-Fu%UNITS_PATH% -Fi%INC_PATH%

%FPC_COMMAND% -Us win32\system.pp
%FPC_COMMAND% win32\classes.pp
%FPC_COMMAND% packages\rtl-extra\winsock.pp
%FPC_COMMAND% packages\winunits-base\comobj.pp
%FPC_COMMAND% packages\winunits-base\shellapi.pp
%FPC_COMMAND% packages\winunits-base\wininet.pp

%FPC_COMMAND% win\messages.pp
%FPC_COMMAND% packages\winunits-base\richedit.pp
%FPC_COMMAND% packages\paszlib\paszlib.pas


%FPC_COMMAND% inc\fpintres.pp
%FPC_COMMAND% win32\sysinitpas.pp



_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/lis

[Jonas Maebe via fpc-devel]

On 24/05/2021 12:18, Yuriy Sydorov via fpc-devel wrote:
> You can't first pass the -Mdelphi switch and then override it with
> -Mfpc. -Mdelphi implicitly enables lot of other compiler options which
> are not reverted by -Mfpc.

That feels like a bug.


Jonas
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/m

[Yuriy Sydorov via fpc-devel]

On 24.05.2021 23:16, Jonas Maebe via fpc-devel wrote:
> On 24/05/2021 12:18, Yuriy Sydorov via fpc-devel wrote:
>> You can't first pass the -Mdelphi switch and then override it with
>> -Mfpc. -Mdelphi implicitly enables lot of other compiler options which
>> are not reverted by -Mfpc.
>
> That feels like a bug.

I've double checked this and the mode change seems to work fine.
The source of the issue is that specifying -Mfpc enables inlining which triggers a bug in the Windows RTL.
I've fixed the bug in r49400.

Yuriy.
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc

[Jonas Maebe via fpc-devel]

On 25/05/2021 18:08, Yuriy Sydorov via fpc-devel wrote:
> I've double checked this and the mode change seems to work fine.
> The source of the issue is that specifying -Mfpc enables inlining which
> triggers a bug in the Windows RTL.
> I've fixed the bug in r49400.

Thanks!


Jonas
_______________________________________________
fpc-devel maillist - fpc-***@lists.freepascal.org
https://lists.