Archive for the ‘Delphi’ Category

JIRA is Down

Thursday, May 16th, 2019

Just a note to EMBT customers, JIRA has been down for several hours now.  It is currently giving the following message

HTTP Status 500 - org.ofbiz.core.util.GeneralRuntimeException: Could not determine database type. (Network error IOException: Connection refused: connect)

With a Java callstack if you like that kind of thing.  EMBT has been advised and I’ve been told EDN may also be affected. Hopefully it will be back up shortly.

More Persistence with Spring4D ORM

Wednesday, May 15th, 2019

Today I decided to ensure the project I am working on is in fact database independent. The best way I have found is to make sure you develop and test on multiple databases as you go. Since the ORM portion of Spring4D often called Marshmellow is supposed to isolate your application from the underlying data store, I figured it would be a piece of cake.

The Spring4D ORM supplies numerous database adapters including UIB (Unified Interbase). There is also another I found for IBX. I decided to try UIB since I want minimal deployment dependencies and am using Firebird, but was a little concerned since the last commit was from Jun 13, 2016.

After pulling down the source code from the repo I discovered that there are no projects for Delphi Rio, so I copied the projects for the last version (21) and renamed them to create Rio projects. Getting the components installed proved quite easy, but once I had the basic data ORM setup code for an alternate database incorporated into my project, that’s when the fun started.

Initially UIB would not load the GDS32.DLL from c:\Windows\System32 that was generated from my Firebird 3.0 Win64 install. Providing the full path did not resolve the issue, and GetLastError() wasn’t helpful so I changed the client library to fbclient, and the app crashed a little further down the line.

The ORM uses a TDatabaseManager class to build the database structures from the registered entity metadata when you call the BuildDatabase method.  It worked for SQLLite, but did not for Firebird as the database file did not exist. Unfortunately the TDatabaseManager was never designed to be extended. The class contains only private fields with no protected accessors, and the constructor as well as the BuildDatabase method, are not virtual. As a result I wrote the following decorator class to provide the extended functionality:

type
  TUIBDatabaseManager = class(TObject)
  private
    FConnection :IDBConnection;
    FUIBConnectionAdapter :TUIBConnectionAdapter;
    FDatabaseManager :TDatabaseManager;
    procedure CreateTheDatabase;
  public
    constructor Create(const connection: IDBConnection);
    procedure BuildDatabase;
  end;

implementation

uses
  Spring.Persistence.SQL.Interfaces, {for TQueryLanguage}
  uiblib;

procedure TUIBDatabaseManager.BuildDatabase;
begin
  //if the database does not exist then create it
  if not FileExists(FUIBConnectionAdapter.Connection.DatabaseName) then
    CreateTheDatabase;

  FDatabaseManager.BuildDatabase;
end;

constructor TUIBDatabaseManager.Create(const connection: IDBConnection);
begin
  FConnection := connection;
  FConnection.QueryLanguage := qlFirebird;
  FDatabaseManager := TDatabaseManager.Create(FConnection);
  FUIBConnectionAdapter := connection as TUIBConnectionAdapter;
end;

procedure TUIBDatabaseManager.CreateTheDatabase;
begin
  FUIBConnectionAdapter.Connection.CreateDatabase(TCharacterSet.csWIN1252);
end;

After I got the database created, the application bombed because it was attempting to create boolean object fields as BIT database columns. Tracing through the code I found the where the datatype mapping was performed and added a snippet for FireBird 3 support after descovering that UIB already supported the Boolean datatype for Interbase and Firebird. All I had to do was compile UIB with the FB30 directive. Here is the amended version of the GetSQLDataTypeName method:

function TFirebirdSQLGenerator.GetSQLDataTypeName(
  const field: TSQLCreateField): string;
begin
  Result := inherited GetSQLDataTypeName(field);
  {add support for Firebird 3.0/Interbase 7 new boolean datatype}
  {$ifdef FB30}
  if (field.TypeInfo.Kind = tkEnumeration) and
     (field.typeInfo = System.TypeInfo(Boolean)) then
    Result := 'BOOLEAN'
  else
  {$endif}
  if StartsText('NCHAR', Result) then
    Result := Copy(Result, 2, Length(Result)) + ' CHARACTER SET UNICODE_FSS'
  else if StartsText('NVARCHAR', Result) then
    Result := Copy(Result, 2, Length(Result)) + ' CHARACTER SET UNICODE_FSS';
end;

Once I got that issue resolved the app crashed on a SQL snippet I had in a call to the ExecuteSQL method because Firebird likes quoted identifiers when you use mixed case column names. Once I had worked around that I found that Firebird threw many more errors importing data due to my Column attributes than SQLLite had. It also complained because I had used a reserved word in Firebird as a table name. After a some more fixups I had a running application capable of using either database back end. In the process I re-discovered how much I like Firebird, even if some of it’s SQL error messages are rather cryptic. It is also much faster importing data that SQLLite.

Persistence with Spring4D

Tuesday, May 14th, 2019

My blog tagline is certainly no accident.  I have been interested in persistence frameworks for a long time, and thought I would use Spring4D’s Marshmellow ORM for a project.  Spring4D has been around for quite some time and just had it’s first conference in Italy so I figured the framework was mature and warranted a closer look.  I had previously used the Spring4D DI container, and decided this time to use Spring4D collections as well to avoid as much code bloat as possible.

The first thing I discovered is that the Spring persistence layer aka Marshmellow has a very unfortunate name.  Trying to google it with “delphi” leads to all sorts of hits related that Android version.  The next thing I learned is that development has been put on hold as of September 2018 due to a lack of resources.  This is unfortunate, as there are not that many open source ORMs that use the newer language features of Delphi when compared C# or Java.

The third thing I quickly learned is that other than the Tests, there is not a whole lot of documentation available.  There is the reference help which is really not that helpful.  It doesn’t contain descriptions of the class interactions or architecture and has no examples of usage.  Really it’s not much more informative than drilling through the code.  The best source of “getting started” help that I could find is the previous bitbucket repo. There is of course also the google groups if you need to get clarification of something, and reading previous posts can help you from running into common issues.

As a newbie, it’s not clear how the [InheritenceAttribute()] works so I will have to investigate it further. I assume if the last descendant in a class hierarchy is the only one marked as an entity, it will by default have all the fields marked with the Column() attribute in all ancestor classes.

Another thing that is not particularly clear from any documentation is how the underlying datatype employed by the database is determined.  There are really only 3 pertinent parameters supplied to the Column() attribute, namely length, precision and scale.  Length only applies to string types and precision, scale to numeric.  It is unclear how precision and scale effectively change the numeric datatype and it’s corresponding precision or scale, other than for Integer datatypes you specify 0,0 for precision,scale.

I’m sure all this will become clear as I use the framework more, but some more thorough ‘Getting Started’ documentation would have been really nice.

How to get Login Dialogs Appearing on the TaskBar

Thursday, May 9th, 2019

Most of the applications I have worked on in Delphi are database apps that may present a splash form quickly followed by a login dialog.  If the user fails to authenticate, the application needs to terminate gracefully.  The only way to do so cleanly is to modify the DPR code with some conditional logic. I’ve seen scenarios where after the main form was created the login dialog was invoked and if authentication failed everything was torn down. This complicates the shutdown logic, and often didn’t work well, encouraging a call to Halt() and sometimes leaving the process in memory.

Any long time Delphi user knows that messing with the generated DPR code in Delphi can cause all sorts of grief later when Delphi tries to auto create forms and add units to the uses clause.  That is out of scope for this post, suffice to say that it is possible to write something like this:

var
  User :TUser;
begin
  Screen.Cursor := crAppStart;
  try
    Application.Initialize;
    Application.MainFormOnTaskbar := True;
    Application.Title := 'My Secure App';
    Application.CreateForm(TMainDm, MainDm);
  finally
    Screen.Cursor := crDefault;
  end;
  User := TfrmLogin.Login
  (
    function (const UserName,Password :string) :TObject
    begin
      Result := MainDm.Session.FindWhere<TUser>(
        Restrictions.&And(
          Restrictions.Eq('UserName',UserName),
          Restrictions.Eq('Password',Password)
        )
      ).FirstOrDefault;
    end,
    {
      UserName can be passed as first parameter so don't
      have to type it in all the time
    }
    ParamStr(1),
    3  {credential retries available }
  ) as TUser;
  if User <> nil then
  begin
    Application.CreateForm(TfrmMain, frmMain);
    frmMain.CurrentUser := User;
    Application.Run;
  end;
end.

The problem is that the Login Dialog does not appear on the Windows Taskbar. If it is hidden behind other windows, the user may think the application has not been launched and attempt to start another instance. There is no easy way for the user to bring the login dialog to the foreground short of closing other windows that may be in front of it. Putting the form on the taskbar solves this. As a quick solution I looked at the SetMainForm method in the Vcl.Forms unit, and decided to extract the ChangeAppWindow() procedure since it is not available outside of the Forms unit. Then I simply called it from this event, and voila! a taskbar button showing the Login form.

procedure TfrmLogin.FormCreate(Sender: TObject);
begin
  ChangeAppWindow(handle,True,True);
end;

I’m sure there are reasons why this method is not exposed as a public TApplication class procedure, but perhaps it could be with a usage caveat.

The Web vNext

Thursday, October 25th, 2018

I have been reading a lot about Web development lately, and pondering the future of web development.  For the longest time I shied away from Web development, largely because it was so laborious and frustrating.

I dabbled a bit back when people were using COM objects with VBScript on the server and Javascript on the client in classic ASP pages. Even back then, companies were making software look like Windows applications within the browser (IE).  By today’s standards, those applications look dated, but today’s web SPAs function in a similar manner, although great progress has been made in standardizing browsers, abstracting out their idiosyncrasies using frameworks, and handling variable display sizes.  My current experience is that it’s still harder and more time consuming to develop a good web application, and they are still not as rich as a Windows app IMHO.

That begs the question as to why people are writing applications for the web anyway.  For eCommerce sites, I get it, but for other line of business applications why not throw off the shackles of HTML/CSS and embrace a new UI framework altogether?  Now I know what you’re thinking…..this guy has lost it and wants to re-invent the web which a lot of smart people have worked on for years to make Web 2.0 a reality.  Before you call the white coats, read on…

When a user goes to a web site for any commercial activity they should be ensuring the website has a valid SSL certificate from a trusted authority.  The Internet is a dangerous place these days where web sites may be trying to attack your computer.  We download content from such sites into our sand boxed browser. This is not that much different than downloading a signed application.  We use signed applications all the time now, when using applications on our phones, tablets and desktops from the “app stores”.  The only difference is whether the store owner i.e.: Apple, has reviewed the application testing for malware.  Open source software such as Tortoise Git is often signed as well to ensure users trust that the application they are getting is from the advertised source and is safe to use.  I think it’s safe to say that most people would trust a signed application.

With that premise in hand, why aren’t we all writing signed Desktop apps using REST back ends?  Or if we really want to leverage single source cross platform applications, why not use WASM with a UI framework that allows an application to be written in languages that are typically used for native desktop, and mobile devices such as C#, Delphi, or C++?

The performance of even Javascript is pretty decent.  Many games have been ported to the browser using Javascript transpilers or EmScripten.  WASM allows developers to take this one step closer to the metal (CPU) and skip the run-time parsing and execution of Javascript.  Blazor is a project that does just that, within the confines of HTML/CSS, but also shows the possibility of using WASM with a different presentation framework such as Uno, or Ooui.  Obviously I’m not the only one thinking that a different presentation layer might be overdue in the web space.

One of the advantages of Blazor is to eliminate some of the third party dependencies, making the development stack more reliable, and to use the same technology and tools throughout.  Of course you can learn and use Javascript throughout the entire stack now, or a Javascript transpiler, but without WASM using your development language of choice is not possible.

I would like to see Object Pascal support the entire development stack.  Preferably the same language dialect and core libraries.  Perhaps something similar to Blazor, or maybe FMX targeting the web with WebGL.  All we need is to be able to capture the compiler IR and feed it into the WASM LLVM back end (okay there might be a little more to it than that).  The web is a huge horizontal market that is ripe for disruption and with the right moves, Delphi could grab a chunk of that segment, making the product relevant again.

What do you think is the future of Javascript on the web?  Will it be dethroned at some point by WASM? Is the future of the Web HTML/CSS, a different UI layer, or perhaps a mix depending on the type of web app?  Should I dust off the Delphi .NET compiler to generate MSIL to feed the Mono WASM run-time, or do we need a way to get the current compilers to output LLVM IR?

Maybe it’s just time to call the white coats as pondering all the options can drive you crazy…

Using a K750 Mac Keyboard with VMWare Fusion

Thursday, August 23rd, 2018

I just created a VM for the Beta test and decided I am going to use it on my Mac for a change. Rather than using a KVM I bought a Logitech M510 mouse, and K750 PC keyboard and bound it to one unifying receiver. Normally I just pull the USB receiver from my PC, put it into my Mac mini, switch the input on my display and I’m off to the races. This is not ideal in that some Mac keyboard mappings are then unavailable to me, but since I mainly run Windows VMs, that was okay until a leg broke on my PC keyboard.

I decided that I would start using my Mac K750, but re-discovered that the function keys are by default mapped to Mac OS/X special functions, and to use the function keys I had to press the fn + F which is a pain because of the location of the fn key and my muscle memory.

I tried using the tips on Malcolm Groves blog but without success as I am running High Sierra. I even installed Karabiner to remap the function keys to normal F keys and that didn’t seem to work for me. What did work was installing the Logitech Control Center. I didn’t have to re-map any keys it just worked…

I of course have VMWare Fusion 10’s Windows 10 Profile Keyboard mapping set to disable Mac OS Host keyboard shortcuts.

Now the only issue I have is that the Logitech M510 mouse does not track very well, and sometimes appears to lose it’s connection to the receiver.  If I lift it up and put it back down, I get my mouse cursor back, but it’s so frustrating I have started using an old Bluetooth Apple Mouse I bought at a garage sale for 25 cents.  Apparently Logitech is known for poor Mac support.

Why I Use(d) Still Use Delphi

Thursday, July 26th, 2018

I started using Object Pascal (Delphi) because it was VB like without the ugly syntax of VB, and it had the power of C++.   Namely that you could create as well as consume COM objects and it was natively compiled with at the time a very fast compiler when compared to C++.  The VCL was a stable UI framework that was simple to use and flexible enough that unusual use cases were still possible.

Sadly, now I use Delphi less and less and for maintenance work only, or personal and open source projects.  The community has dwindled from what it was once, local jobs are harder to find, and many luminaries have been lost to the evil empire (M$) and it’s technology.

I must admit that I am now working predominantly on C# .NET Core projects. I would gladly come back from the dark side if someone wanted to entice me with a interesting Delphi project. I find C# syntax more terse and unreadable than Object Pascal, but there is no mistaking the power of LINQ, the PPL, and the abundance of open source projects and technical resources. .NET native also makes adoption of C# even more tempting. If they can write Kestrel in C# you should be able to write anything that requires good performance in C#. I would love to see some Delphi web server benchmarks and how they compare to Kestrel. Favorable results might generate more interest in Delphi as would compiler benchmarks.

I love Delphi. I wish it a bright future.  Without new developers it will most certainly fade away over time.  New developers shy away from languages that won’t get them jobs.  An investment in learning a language, an IDE, and run-time libraries simply has to pay off.  Without a growing market share, expensive tools have to provide developers with a competitive edge. For this reason, I applaud EMBT’s release of the Community Edition. I hope it will bring new blood into the development community and spark more open source projects. There are some amazing open source projects out there that could use Community Edition to further their efforts.

There are also many exciting things happening in the land of Pascal that I hope will expand the community. REMObjects has released their Oxygene compiler that can also target WASM as well as native code. Smart Mobile Studio has released version 3 of their Pascal language targeting Javascript, and TMS has introduced a similar product called WebCore based on the FPC Javascript transpiler.  EMBT has hired Jon Aasenden and purchased Sencha so who knows what they have planned. With UniGUI and other frameworks Object Pascal (or rather different dialects) have many more options to target the web than only a couple years ago. I long for the day when I could use the same Object Pascal language for web, desktop, mobile and server development without compromising the results. Is it time for a language standard?

Is this Feature Supported?

Monday, January 22nd, 2018


Quite often new features are added into a code stream for demonstration purposes, or need to be hidden because they are not quite ready for release (especially true for monolithic EXEs), or the client cannot upgrade their database structures yet.

There usually are various home grown approaches to accomplish this ranging from command line parameters (beta switches to enable or disable functionality), to one off database metadata queries.  These approaches can be unified in an interface added to your database “data session” as shown in the following code snippets.


type
  TFeatureSupportedPredicate = reference to function(DataSession :IDataSession): Boolean;
  TFeatureInfo = class(TObject)
    Supported :integer;
    FeatureSupportedPredicate :TFeatureSupportedPredicate;
  public
    constructor Create;
  end;

  constructor TFeatureInfo.Create;
  begin
    inherited;
    Supported := -1;  //used to indicate the test for existence has not been performed
  end

type
  TDataSession  = class(TInterfacedObject,IDataSession)
  var
    FFeatureSupportDictionary :TDictionary<string,TFeatureInfo>;
    constructor Create; override;
  end;


  constructor  TDataSession.Create;
  begin
    //do some stuff
    FFeatureSupportDictionary := TDictionary<string,TFeatureInfo>.Create;
  end;

procedure TDataSession.AddFeatureSupportPredicate(
            const FeatureName :string;
            Predicate :TFeatureSupportedPredicate);
var
  FeatureInfo :TFeatureInfo;
begin
  if FFeatureSupportDictionary.ContainsKey(FeatureName) then
    raise EFeatureSupportException.
                      CreateFmt('A Feature Support Predicate is already '+
                                'registered with the Name: ''%s'' ',[FeatureName]);
  FeatureInfo := TFeatureInfo.Create;
  FeatureInfo.FeatureSupportedPredicate := Predicate;
  FFeatureSupportDictionary.Add(FeatureName,FeatureInfo);
end;

function TDataSession.IsFeatureSupported(const FeatureName :string) :boolean;
var
  FeatureInfo :TFeatureInfo;
begin
  Result := False;
  if FFeatureSupportDictionary.TryGetValue(FeatureName,FeatureInfo) then
  begin
    if FeatureInfo.Supported = -1 then
      FeatureInfo.Supported := ifthen(FeatureInfo.FeatureSupportedPredicate(Self),1,0);
    Result := Boolean(FeatureInfo.Supported);
  end;
end;



Now wherever you pass or inject your IDataSession interface, which in a Client/Server application is almost everywhere, you will have access to the FeatureSupport functionality, and you can add/test predicates with the following code snippets:

  AddFeatureSupportPredicate(SomeFeature,
    function(DataSession: IDataSession): Boolean
    begin
      Result := DataSession.TableExists('SomeTableForTheFeature');
    end);

if DataSession.IsFeatureSupported(SomeFeature) then ....

If Civil Engineering was like Software Engineering

Wednesday, December 13th, 2017

We all know that software engineering is not at all like any other engineering discipline.  It is both a creative endeavor and an expression of technical acumen.  Applying the term “engineer” to this career is somewhat akin to the title “Domestic Engineer”.  If wasn’t, software engineers would not take an initial design, and then change the design after 90% of it was implemented as a SOP (standard operation procedure).  If civil engineering did such a thing you would see a lot more bridges collapse, and buildings like this one

Plans Change

Plans Change

Stringification

Monday, December 4th, 2017

String handling in Delphi has always been quick, but this has lead to the abuse of the data type.  I have seen code where strings are used to store all kinds of data (ie: Float) and is continually converted back and forth depending on whether it’s needed for the UI, a calculation, or to generate dynamic SQL.  Whenever data is stringified, it is open to interpretation by all consumers of that data.  Should it be handled as a string or converted to another data type and if so what type with what kind of precision and scale is necessary to avoid loss?  Who is responsible for conversion (the caller or the callee)?

By not using the native type for a given piece of data it becomes unclear as to how the data should be transformed in order for it to be stored for example to disk in the form of a config file entry, or used as a string in a SQL clause built using the Format() function.  Environmentally sensitive functions like Format() can cause massive code breakage when the user changes for example the decimal separator character, resulting in invalid SQL statements being generated.

The identifier name used for fields or variables representing data from a database column may be re-used to identify the stringified fields, causing confusion and compilation errors over the data type differences if hungarian or some other notation is not used as part of variable names.

Once data is stringified, there is no longer a data contract, and what ensues is a lot of wet (aka not DRY) brittle code (repeated data conversions) thrown together in a haphazard fashion in an effort to just “make it work”.  Making little changes like supporting different locales with say a comma instead of a decimal point for float values becomes a painful experiment if the applications are not properly layered.

My advice; write true native code, keeping fields and variables in their native type, and avoid stringification as much as possible.