Compile Tool #1 – An easier way to compile projects

Some people may have wondered if I have fallen off the face of the earth as I have been less vocal the last couple of weeks.

It has nothing to do with the dreadful COVID-19 infection I suppose most of us, one way or the other, are affected by. It rather has to do with being overly busy with various things.

One of the things, that relate to kbmMW and kbmMemTable, is the development of a brand new CompileTool which will be included in next release of kbmMemTable and kbmMW.

The purpose of the CompileTool is … TA DAAA…. to compile stuff 🙂

So what’s new about that? Not really much… but let me explain the rationale behind my apparant brain damage.

The Compile Tool

Compiling and installing kbmMemTable, has in Delphi always been fairly easy. In C++Builder only mode, not so much, partly because the C++ only environment diverge more and more from what kbmMemTable originally supported, and the matching C++ project files.

To complicate matters even more, kbmMW can be a pain to install in Delphi due to kbmMW’s ability to seemlessly integrate with loads of 3rdparty stuff. Paths and requirements and more needs to be provided. In C++ only mode it is even more complex to get it going, and despite the Compile Tool helping much on the situation, it is not fully solved with kbmMW yet, because C++Builder exhibits random crashes and unexplained compile/link errors (internal errors).

But one of the things Delphi did very well.. in the early days (I suppose until XE got to see the light), was to consistently tell you that some 3rdparty packages should be referenced to compile kbmMW’s packages nicely. Delphi was even nice enough to add the relevant requirements to the kbmMW package so everything just worked.

Unfortunately, Delphi has stopped to do that stabily since many years. I have reported it to Embarcadero on numerous occations, and they have acklowledged the issue, but have not been able to figure out why it has stopped working. Mind you.. sometimes it works… but then suddenly it does not, usually when that happens, it stops working for good in my experience.

So the Compile Tool has as goal to:

1. Know everything about the units and requirements of the project in hand
2. Be able to manage knowledge about 3rdparty libraries/packages
3. Produce correct valid project files for the project that the Compile Tool has been prepared for.
4. Compile and install the projects automatically
5. Be able to recompile and restart itself, so it is up to date with whatever settings you may have made in for example kbmMWConfig.inc and thus based on those settings, is able to produce correct project files.

At first I made it to make compilation and installation of kbmMW easier, but it soon dawned to me that kbmMemTable should be supported too (standalone) and that it as such could be a generic tool that will work for other developers too. (Currently it is not released with license for other 3rdparty developers to use it, but ping me if you have interest in that.)

Compile Tool for kbmMemTable

Let us have a look at the Compile Tool for kbmMemTable. It will usually be found, as CompileTool.exe, in the source directory of the project for which it is supposed to support. Further the source of the Compile Tool will be found in the subdirectory CompileTool under the managed projects source directory.

If you loose CompileTool.exe, it can be recompiled by opening and building the project in the CompileTool directory. You can not use the CompileTool source/executable from a different project (like kbmMW), because they contain different settings, specially in the uCompileToolFeatures.pas file, which is specialized for each project.

If you start CompileTool.exe on a computer on which Delphi, C++Builder or RAD Studio is not installed, you will get an exception and the tool will shut down.

So let us start it on a computer with RAD Studio installed.

On the left side, a list of supported features for the project, is listed. In the case of kbmMemTable it is pretty simple, and will not change, since all features are available for all versions of kbmMemTable. However look later for how the Compile Tool for kbmMW looks.

In the center, the discovered versions of Delphi, C++Builder or RAD Studio is listed. You can select, in which of them, kbmMemTable should be installed.

On the right side, you can follow the status of what is happening.

At the bottom, various buttons are available.

Update compile tool – Will rebuild the Compile Tool itself, and restart it with the newly compiled executable. For kbmMemTable it is not often needed to do, but in the case of kbmMW, you will want to do that, everytime you have changed something in kbmMWConfig.inc.

Validate requirements – It will show a dialog, where 3rdparty requirements, which the Compile Tool has not been able to resolve itself, can be defined. The definitions made here will be remembered for next time in the file CompileTool.ini, making it easier to recompile/install without having to reconfigure each time. The .ini file will never be overwritten by kbmMemTable or kbmMW installations. kbmMemTable will usually not need any settings in this dialog. See section about kbmMW installation further down for more information.

Generate projects – It will produce new kbmMemTable project files matching the selected IDE. The project files will automatically be written to the parent directory (which is the kbmMemTable source directory).

Generate, compile and install – It will generate projects, as described above, compile the projects and automatically install the resulting packages in the IDE, for all selected IDE’s. Usually you will be required to close the selected IDE before being able to compile and install. You can start with the option -F:  CompileTool.exe -F to override the requirement to stop the IDE. However the packages will not show up until you restart the IDE later on, and if the packages already was in use by the IDE you will get compile/linker errors.

This is an example the result. You may notice that there are various paths shown in the status. Those paths are automatically picked up from your current installation, and provided for the compiler by the Compile Tool.

After it succeeds, you can close the tool, and start the IDE. Now kbmMemTable will be available and installed with all paths for Windows 32 compilation, correctly setup automatically.

So let us look at how it works when installing kbmMW.

Compile Tool for kbmMW

In this case, I have, for the demo, opened kbmMWConfig.inc, uncommented the line:

{$DEFINE KBMMW_DBISAM3_SUPPORT} // DBISAM 3 support.

and saved the file again, which essentially tells kbmMW that we want full support for DBISAM v3. (FTR there are similar defines for 36 other databases too, incl. DBISAM v4, ElevateDB, NexusDB and many many more).

Since this is a new setting, I first start the Compile Tool, where the Feature support at this time do not include DBISAM3, and let it recompile itself.

When it restarts, it looks like this:

Now the Compile Tool recognize the request for feature support for ElevateSoft DBISAM v3.
Since it is a new 3rdparty tool that kbmMW should support, compared to previous settings, we want to check the requirements dialog by clicking on the Validate requirements button.

You can see that there are a runtime package requirement that is currently unresolved. To resolve it, simply type the name of the DBISAM v3 runtime package, typically something like db300d20 or something along those lines. As the structure of the package name can vary wildly between various 3rdparty projects, it is left for you to type the right value. Remember that this value will be used for all the IDE’s that the Compile Tool will compile for.

You can include parameters in the name. Eg. DBISAM3=db300[!–IDESHORTYPE–!][!–IDESHORTID–!] which will replace [!–IDESHORTYPE–!] with D for Delphi or C for C++Builder and [!–IDESHORTID–!] with 20 for Delphi 10.3.

If you need to refer to multiple packages/requirements for the DBISAM3 selection, you can separate those with a semicolon ;

Click save, and the Compile Tool will remember your settings, also for next time you start the Compile Tool.

Clicking either Create projects or Create, compile and install, will ensure the kbmMW project files contains the relevant requirements.

Also notice that there is an extra Prerequisites section at the top of the Compile Tools window. It is there because kbmMW require compilation and installation of kbmMemTable beforehand. You can point out where it’s source is, and click the checkbox, then it will automatically recompile and install it when you recompile and install kbmMW via the Create, compile and install button.

Behind the scenes

So what is happening behind the scenes? Well… I told about the uCompileToolFeatures.pas file which is special for each project. It is in that file of the Compile Tool sources, where the project specialities are defined.

type
  TkbmCTFeatures = class
  const
     LOWEST_SUPPORTED_BDS_VERSION = 12.0; // XE5
  public
     class function GetText(const ATextInfo:TkbmCTTextInfo):string;
     class procedure RegisterRuntimeFeatures(const AInfo:TProjectInfos);
     class procedure RegisterDesigntimeFeatures(const AInfo:TProjectInfos);
     class function BuildParameters(const AMain:TfrmMain; const ACpp:boolean; const AIDE:TIDEInfo):TStringList;
     class function GenerateProjectFileName(const ACpp:boolean; const AIDE:TIDEInfo; const ADesignTime:boolean):string;
  end;

It contains a class definition which groups a few methods which should be defined for a project.

class function TkbmCTFeatures.GetText(const ATextInfo:TkbmCTTextInfo):string;
begin
     case ATextInfo of
          cttiCaption:                 Result:='kbmMW Compile Tool';
          cttiRebuildToolCaption:      Result:='Update this compile tool by automatically recompiling it. It is required if kbmMWConfig.inc has been modified since last time the compile tool was compiled.';
          cttiRecreateProjectsCaption: Result:='Recreate kbmMW project files from scratch based on settings in kbmMWConfig.inc';
          cttiRecreateInstallCaption:  Result:='Recreate kbmMW project files from scratch based on settings in kbmMWConfig.inc,, compile and install if possible';
          cttiPrerequisiteCaption:     Result:='Auto compile and install kbmMemTable';
          cttiPrerequisiteExplanation: Result:='If checked, select path to kbmMemTable source and project files';
     end;
end;

The above specifies the texts to be shown and thus can be configured for other projects (like kbmMemTable which Compile Tool has a similar section).

class function TkbmCTFeatures.BuildParameters(const AMain:TfrmMain; const ACpp:boolean; const AIDE:TIDEInfo):TStringList;

This method builds relevant parameters that must exist for the project file generation. It includes version numbers, project names and descriptions and more. The method will be called multiple times during project generation.

class procedure TkbmCTFeatures.RegisterRuntimeFeatures(const AInfo:TProjectInfos);
begin
     AInfo.AddProjectInfo('RTL','Embarcadero RTL','rtl','',true);

     AInfo.AddProjectInfo('VCL','Embarcadero VCL','vcl;vclimg','',true);

     AInfo.AddProjectInfo('FMX','Embarcadero FMX','fmx','',true);

{$IFDEF KBMMW_ENTERPRISE_EDITION}
     AInfo.AddProjectInfo('KBMMEMTABLE','kbmMemTable Professional Edition','kbmMemRun[!--IDE--!]Pro','',true);
{$ELSE}
 {$IFDEF KBMMW_PROFESSIONAL_EDITION}
     AInfo.AddProjectInfo('KBMMEMTABLE','kbmMemTable Professional Edition','kbmMemRun[!--IDE--!]Pro','',true);
 {$ELSE}
     AInfo.AddProjectInfo('KBMMEMTABLE','kbmMemTable Standard Edition','kbmMemRun[!--IDE--!]Std','',true);
 {$ENDIF}
{$ENDIF}
...
 {$IFDEF KBMMW_DBISAM3_SUPPORT}
     AInfo.AddProjectInfo('DBISAM3','ElevateSoft DBISAM v3 data adapter','?','kbmMWDBISAM3',true);
 {$ENDIF}

...

This section defines all the features that can exist in a kbmMW runtime package, and their library requirements and units, including the DBISAM v3 option.

AddProjectInfo takes 5 arguments:

1. The unique ID of the project part. For example KBMMEMTABLE. Any ID can be used, as long as it is unique.
2. The descriptive name of the project part.
3. The libraries that are required for the project part, separated by semicolon and without file extensions. If it is an empty string, there are no requirements for that particular project part. If it is a question mark, it is unknown, and thus can be handled by the user in the Compile Tool package resolver dialog. It is allowed to include paths if needed, but recommended to only use relative paths from the Source directory.
   Further it is legal to prefix each library with either < or }. Doing so will ensure to sort the item first (<) or last (}), rather just according to its regular name, when project files are generated.
4. The unit names (without extension) that are to be part of this project part. Multiple unit names can be specified separated by semicolon (;). If the unit also encompasses a form or datamodule file, use this syntax:
   unitname=formname:formclass. Eg. kbmMWCustomJavaService=kbmMWCustomJavaService:TkbmMWSimpleService;}JNI
5. A boolean indicating if a requirement is mandatory for this project part. It is used for validation/warning that the project file may not have been generated correctly, if the requirement value has not been made available.

class procedure TkbmCTFeatures.RegisterDesigntimeFeatures(const AInfo:TProjectInfos);
begin
     AInfo.AddProjectInfo('RTL','Embarcadero RTL','rtl','',true);

     AInfo.AddProjectInfo('VCL','Embarcadero VCL','vcl;vclimg;vclx','',true);

     AInfo.AddProjectInfo('IDE','Embarcadero IDE','designide','',true);

     AInfo.AddProjectInfo('FMX','Embarcadero FMX','fmx','',true);

{$IFDEF KBMMW_LICENSE_DATABASE}
     AInfo.AddProjectInfo('KBMMW core database','kbmMW Core database','dbrtl;vcldb','',true);
{$ENDIF}
...

This section defines all the features that can exist in a kbmMW designtime package, and their library requirements and units.

It follows the same explanation as for the runtime part.

class function TkbmCTFeatures.GenerateProjectFileName(const ACpp:boolean; const AIDE:TIDEInfo; const ADesignTime:boolean):string;
...

The above code is used for creating the relevant project file name for a specific IDE. The file name should not include any file extensions. The method will be called multiple times during project creation.

Final plea

If you have reached here, then you are qualified to assist me making the Compile Tool better. Since kbmMW supports so many 3rdparty tools, I have for now only specified library requirements in the Compile Tool for some of them. For the remaining, I have left it for you to define, simply because I do not know the naming rules for all the various 3rdparty libraries. Please comment here on this thread if you have some pet libraries that kbmMW supports and that you would like the Compile Tool to know about, preferably with the complete description of how the library name is defined. I.e. what each part of the name consists of.

If the name do not change based on the version of the 3rdparty library, it most likely will be possible to add automatic support in the Compile Tool for that particular library, making it even easier to compile and install kbmMW.