TPersistent.com

I’m relatively new to the Android development platform, coming from the Windows desktop. One of the great things about recent editions of Delphi is it’s ability target numerous other platforms. One of the worst things about recent versions of Delphi is it’s ability to target platforms you are not as familiar with. If your use case is atypical, you can quickly find yourself swimming in shark infested waters. You never know what is going to byte you, a bug in the platform, a bug in Delphi RTL or FMX code, a bug in your code, or just your lack of knowledge about the platform.

Today I was trying to figure out why files I was saving to the Documents folder on my Android LG5 were visible from the Android File Manager, but not present when I connected the phone to a Windows, or Ubuntu desktop. Turns out there is a long outstanding bug in Android that google has not even acknowledged. In order to see new files when connecting to a device using the MTP protocol, you may need to re-boot the Android device before the files appear. Indeed once I did so, my files appeared and I could copy them off to my PC. You may want to comment on this motion to get the issue re-opened and addressed.

Hopefully this helps someone else avoid spending a lot of time scratching their head…

Just a note to EMBT customers, JIRA has been down for several hours now.  It is currently giving the following message

HTTP Status 500 - org.ofbiz.core.util.GeneralRuntimeException: Could not determine database type. (Network error IOException: Connection refused: connect)

With a Java callstack if you like that kind of thing.  EMBT has been advised and I’ve been told EDN may also be affected. Hopefully it will be back up shortly.

Today I decided to ensure the project I am working on is in fact database independent. The best way I have found is to make sure you develop and test on multiple databases as you go. Since the ORM portion of Spring4D often called Marshmellow is supposed to isolate your application from the underlying data store, I figured it would be a piece of cake.

The Spring4D ORM supplies numerous database adapters including UIB (Unified Interbase). There is also another I found for IBX. I decided to try UIB since I want minimal deployment dependencies and am using Firebird, but was a little concerned since the last commit was from Jun 13, 2016.

After pulling down the source code from the repo I discovered that there are no projects for Delphi Rio, so I copied the projects for the last version (21) and renamed them to create Rio projects. Getting the components installed proved quite easy, but once I had the basic data ORM setup code for an alternate database incorporated into my project, that’s when the fun started.

Initially UIB would not load the GDS32.DLL from c:\Windows\System32 that was generated from my Firebird 3.0 Win64 install. Providing the full path did not resolve the issue, and GetLastError() wasn’t helpful so I changed the client library to fbclient, and the app crashed a little further down the line.

The ORM uses a TDatabaseManager class to build the database structures from the registered entity metadata when you call the BuildDatabase method.  It worked for SQLLite, but did not for Firebird as the database file did not exist. Unfortunately the TDatabaseManager was never designed to be extended. The class contains only private fields with no protected accessors, and the constructor as well as the BuildDatabase method, are not virtual. As a result I wrote the following decorator class to provide the extended functionality:

type
  TUIBDatabaseManager = class(TObject)
  private
    FConnection :IDBConnection;
    FUIBConnectionAdapter :TUIBConnectionAdapter;
    FDatabaseManager :TDatabaseManager;
    procedure CreateTheDatabase;
  public
    constructor Create(const connection: IDBConnection);
    procedure BuildDatabase;
  end;

implementation

uses
  Spring.Persistence.SQL.Interfaces, {for TQueryLanguage}
  uiblib;

procedure TUIBDatabaseManager.BuildDatabase;
begin
  //if the database does not exist then create it
  if not FileExists(FUIBConnectionAdapter.Connection.DatabaseName) then
    CreateTheDatabase;

  FDatabaseManager.BuildDatabase;
end;

constructor TUIBDatabaseManager.Create(const connection: IDBConnection);
begin
  FConnection := connection;
  FConnection.QueryLanguage := qlFirebird;
  FDatabaseManager := TDatabaseManager.Create(FConnection);
  FUIBConnectionAdapter := connection as TUIBConnectionAdapter;
end;

procedure TUIBDatabaseManager.CreateTheDatabase;
begin
  FUIBConnectionAdapter.Connection.CreateDatabase(TCharacterSet.csWIN1252);
end;

After I got the database created, the application bombed because it was attempting to create boolean object fields as BIT database columns. Tracing through the code I found the where the datatype mapping was performed and added a snippet for FireBird 3 support after descovering that UIB already supported the Boolean datatype for Interbase and Firebird. All I had to do was compile UIB with the FB30 directive. Here is the amended version of the GetSQLDataTypeName method:

function TFirebirdSQLGenerator.GetSQLDataTypeName(
  const field: TSQLCreateField): string;
begin
  Result := inherited GetSQLDataTypeName(field);
  {add support for Firebird 3.0/Interbase 7 new boolean datatype}
  {$ifdef FB30}
  if (field.TypeInfo.Kind = tkEnumeration) and
     (field.typeInfo = System.TypeInfo(Boolean)) then
    Result := 'BOOLEAN'
  else
  {$endif}
  if StartsText('NCHAR', Result) then
    Result := Copy(Result, 2, Length(Result)) + ' CHARACTER SET UNICODE_FSS'
  else if StartsText('NVARCHAR', Result) then
    Result := Copy(Result, 2, Length(Result)) + ' CHARACTER SET UNICODE_FSS';
end;

Once I got that issue resolved the app crashed on a SQL snippet I had in a call to the ExecuteSQL method because Firebird likes quoted identifiers when you use mixed case column names. Once I had worked around that I found that Firebird threw many more errors importing data due to my Column attributes than SQLLite had. It also complained because I had used a reserved word in Firebird as a table name. After a some more fixups I had a running application capable of using either database back end. In the process I re-discovered how much I like Firebird, even if some of it’s SQL error messages are rather cryptic. It is also much faster importing data that SQLLite.

My blog tagline is certainly no accident.  I have been interested in persistence frameworks for a long time, and thought I would use Spring4D’s Marshmellow ORM for a project.  Spring4D has been around for quite some time and just had it’s first conference in Italy so I figured the framework was mature and warranted a closer look.  I had previously used the Spring4D DI container, and decided this time to use Spring4D collections as well to avoid as much code bloat as possible.

The first thing I discovered is that the Spring persistence layer aka Marshmellow has a very unfortunate name.  Trying to google it with “delphi” leads to all sorts of hits related that Android version.  The next thing I learned is that development has been put on hold as of September 2018 due to a lack of resources.  This is unfortunate, as there are not that many open source ORMs that use the newer language features of Delphi when compared C# or Java.

The third thing I quickly learned is that other than the Tests, there is not a whole lot of documentation available.  There is the reference help which is really not that helpful.  It doesn’t contain descriptions of the class interactions or architecture and has no examples of usage.  Really it’s not much more informative than drilling through the code.  The best source of “getting started” help that I could find is the previous bitbucket repo. There is of course also the google groups if you need to get clarification of something, and reading previous posts can help you from running into common issues.

As a newbie, it’s not clear how the [InheritenceAttribute()] works so I will have to investigate it further. I assume if the last descendant in a class hierarchy is the only one marked as an entity, it will by default have all the fields marked with the Column() attribute in all ancestor classes.

Another thing that is not particularly clear from any documentation is how the underlying datatype employed by the database is determined.  There are really only 3 pertinent parameters supplied to the Column() attribute, namely length, precision and scale.  Length only applies to string types and precision, scale to numeric.  It is unclear how precision and scale effectively change the numeric datatype and it’s corresponding precision or scale, other than for Integer datatypes you specify 0,0 for precision,scale.

I’m sure all this will become clear as I use the framework more, but some more thorough ‘Getting Started’ documentation would have been really nice.

Most of the applications I have worked on in Delphi are database apps that may present a splash form quickly followed by a login dialog.  If the user fails to authenticate, the application needs to terminate gracefully.  The only way to do so cleanly is to modify the DPR code with some conditional logic. I’ve seen scenarios where after the main form was created the login dialog was invoked and if authentication failed everything was torn down. This complicates the shutdown logic, and often didn’t work well, encouraging a call to Halt() and sometimes leaving the process in memory.

Any long time Delphi user knows that messing with the generated DPR code in Delphi can cause all sorts of grief later when Delphi tries to auto create forms and add units to the uses clause.  That is out of scope for this post, suffice to say that it is possible to write something like this:

var
  User :TUser;
begin
  Screen.Cursor := crAppStart;
  try
    Application.Initialize;
    Application.MainFormOnTaskbar := True;
    Application.Title := 'My Secure App';
    Application.CreateForm(TMainDm, MainDm);
  finally
    Screen.Cursor := crDefault;
  end;
  User := TfrmLogin.Login
  (
    function (const UserName,Password :string) :TObject
    begin
      Result := MainDm.Session.FindWhere<TUser>(
        Restrictions.&And(
          Restrictions.Eq('UserName',UserName),
          Restrictions.Eq('Password',Password)
        )
      ).FirstOrDefault;
    end,
    {
      UserName can be passed as first parameter so don't
      have to type it in all the time
    }
    ParamStr(1),
    3  {credential retries available }
  ) as TUser;
  if User <> nil then
  begin
    Application.CreateForm(TfrmMain, frmMain);
    frmMain.CurrentUser := User;
    Application.Run;
  end;
end.

The problem is that the Login Dialog does not appear on the Windows Taskbar. If it is hidden behind other windows, the user may think the application has not been launched and attempt to start another instance. There is no easy way for the user to bring the login dialog to the foreground short of closing other windows that may be in front of it. Putting the form on the taskbar solves this. As a quick solution I looked at the SetMainForm method in the Vcl.Forms unit, and decided to extract the ChangeAppWindow() procedure since it is not available outside of the Forms unit. Then I simply called it from this event, and voila! a taskbar button showing the Login form.

procedure TfrmLogin.FormCreate(Sender: TObject);
begin
  ChangeAppWindow(handle,True,True);
end;

I’m sure there are reasons why this method is not exposed as a public TApplication class procedure, but perhaps it could be with a usage caveat.

During development, you sometimes want to make database changes that a migration cannot handle.  One such example is changing the datatype of a primary key column from an int to bigint.  In such a case you may want to burn the database to the ground and have a single migration that creates all the database structures defined through the model to date.  Some developers routinely remove all migrations to “cleanup” the code.

The procedure to do so is different for EF vs EF Core, and if you don’t do it the right way, things can get screwed up and be hard to fix.  With EF Core the easiest way I have found to burn the database to the ground is to use the PMC command:

update-database 0

This is a little different than simply removing all the database tables from a SQL utility like SMS.  It actually removes all tables, but ensures that the __EFMigrationsHistory table is present with 0 rows.

Then you can create delete all the files in the Migrations folder of the C# project and create a new initial migration:

add-migration "Some Title"

and generate the new structures:

update-database

Recently the hard drive in my Toshiba Satellite gave up the ghost, so I decided to stick in an SSD and update the OS.  Originally the laptop shipped with Windows 8 (yeach!).  The laptop kept hanging as it accessed the hard drive.  I tried repairing it by booting off a Windows CD, but it kept locking up.  Even after installing a new SSD this issue persisted.  I could not get any version of Windows to install.

Finally I decided that I would try to install a Linux Distro.  I had previously used Ubuntu and found it to be reasonably reliable and user friendly,  Googling for Distro rankings I came across Elementary OS.  It’s Mac like UI intrigued me, so I set about installing it.  Unlike Windows, the Linux distro installed without issue on my SSD, and seems quite quick.

I happily discovered that Firefox and Skype are available and work quite well although installing them was a little more complicated than running a Windows installer.  Unfortunately I cannot say the same for VMWare Player, so for now running my VMs on the laptop is not possible.  Between the apps that come bundled, and those available for this Ubuntu based Linux distro, I have everything an average use would need on a daily basis.

Unfortunately I installed Loki (v4) and to “upgrade” to the latest OS version of Juno (v5.0) meant a complete re-install of the OS and apps.  Hopefully future “upgrades” will be available using the AppCentre.

I have been reading a lot about Web development lately, and pondering the future of web development.  For the longest time I shied away from Web development, largely because it was so laborious and frustrating.

I dabbled a bit back when people were using COM objects with VBScript on the server and Javascript on the client in classic ASP pages. Even back then, companies were making software look like Windows applications within the browser (IE).  By today’s standards, those applications look dated, but today’s web SPAs function in a similar manner, although great progress has been made in standardizing browsers, abstracting out their idiosyncrasies using frameworks, and handling variable display sizes.  My current experience is that it’s still harder and more time consuming to develop a good web application, and they are still not as rich as a Windows app IMHO.

That begs the question as to why people are writing applications for the web anyway.  For eCommerce sites, I get it, but for other line of business applications why not throw off the shackles of HTML/CSS and embrace a new UI framework altogether?  Now I know what you’re thinking…..this guy has lost it and wants to re-invent the web which a lot of smart people have worked on for years to make Web 2.0 a reality.  Before you call the white coats, read on…

When a user goes to a web site for any commercial activity they should be ensuring the website has a valid SSL certificate from a trusted authority.  The Internet is a dangerous place these days where web sites may be trying to attack your computer.  We download content from such sites into our sand boxed browser. This is not that much different than downloading a signed application.  We use signed applications all the time now, when using applications on our phones, tablets and desktops from the “app stores”.  The only difference is whether the store owner i.e.: Apple, has reviewed the application testing for malware.  Open source software such as Tortoise Git is often signed as well to ensure users trust that the application they are getting is from the advertised source and is safe to use.  I think it’s safe to say that most people would trust a signed application.

With that premise in hand, why aren’t we all writing signed Desktop apps using REST back ends?  Or if we really want to leverage single source cross platform applications, why not use WASM with a UI framework that allows an application to be written in languages that are typically used for native desktop, and mobile devices such as C#, Delphi, or C++?

The performance of even Javascript is pretty decent.  Many games have been ported to the browser using Javascript transpilers or EmScripten.  WASM allows developers to take this one step closer to the metal (CPU) and skip the run-time parsing and execution of Javascript.  Blazor is a project that does just that, within the confines of HTML/CSS, but also shows the possibility of using WASM with a different presentation framework such as Uno, or Ooui.  Obviously I’m not the only one thinking that a different presentation layer might be overdue in the web space.

One of the advantages of Blazor is to eliminate some of the third party dependencies, making the development stack more reliable, and to use the same technology and tools throughout.  Of course you can learn and use Javascript throughout the entire stack now, or a Javascript transpiler, but without WASM using your development language of choice is not possible.

I would like to see Object Pascal support the entire development stack.  Preferably the same language dialect and core libraries.  Perhaps something similar to Blazor, or maybe FMX targeting the web with WebGL.  All we need is to be able to capture the compiler IR and feed it into the WASM LLVM back end (okay there might be a little more to it than that).  The web is a huge horizontal market that is ripe for disruption and with the right moves, Delphi could grab a chunk of that segment, making the product relevant again.

What do you think is the future of Javascript on the web?  Will it be dethroned at some point by WASM? Is the future of the Web HTML/CSS, a different UI layer, or perhaps a mix depending on the type of web app?  Should I dust off the Delphi .NET compiler to generate MSIL to feed the Mono WASM run-time, or do we need a way to get the current compilers to output LLVM IR?

Maybe it’s just time to call the white coats as pondering all the options can drive you crazy…

Just as a follow up note to my previous post about using my K750 keyboard on a Mac with VMWare Fusion, I mentioned that my Logitech M510 mouse wasn’t tracking all that well.

In fact, sometimes my keyboard would not seem responsive either.  It seemed to be a problem only when using my external USB 3.0 enclosure which houses the SSD I run my Fusion VMs from.  I thought for a while that perhaps I was experiencing some interference between my WiFi and the Logitech Unifying Receiver.  Odd, because I have been using Logitech for a long time at home without issue.  What lead me to believe it might be interference was an experience at work.  When a new WiFi wap was added close to my machine I got similar behaviour from my keyboard and sometimes the Google speakers would cut out while playing music.  When I switched to a Bluetooth keyboard the issue stopped.

As a result, I was looking around for a Bluetooth keyboard and mouse combination that I liked to replace mine.  Unfortunately the only solar keyboard I could find was the Logitech ones I already owned.  Even Logitech for some reason wasn’t making a solar Bluetooth version.  I love never having to charge or replace batteries.  I wish my mouse was solar too!

For kicks, I did some googling to see if other Mac users were having issues, and eventually found links that indicated people had keyboard and mouse issues after upgrading OS/X to High Sierra, which was when my keyboard issues started (although I never connected the dots myself).  One poster had similar problems with a USB device and indicated that for whatever reason, when he plugged the USB device into a hub rather than directly into his Mac mini the problem did not occur.

I dug up a USB hub I bought as part of a tub of miscellaneous cables at a garage sale, plugged my USB SSD and unifying receiver into it and my problems disappeared!

As far as I am concerned the OS/X versions named after the big cats were less problematic.

I have no issue with Microsoft wanting to patch the OS and bundled applications to make them more reliable.  The problem I have is that Windows 10 Updates are always being applied at an inconvenient time.  If you defer an update you have limited choices, and more often than not, Windows just re-starts on it’s own to apply them without me even knowing one was pending.  I have a bad habit of keeping scratch notes in a NotePad++ file and not saving it.  More than once Windows has re-started and I’ve lost those notes.

The worst thing is, it can take Windows a long time to apply an update and in the mean time you’re twiddling your thumbs.  What was wrong with the way updates have been handled for the last ~20 years?  Worse yet, some of those Windows Updates have caused issues, and required rollbacks.

We all know software is more complex than ever and hard to get right, so why would Microsoft force updates upon their user base?  At least let the user have the illusion they are in control of their own computer.  Apple notifies of updates for Mac OS/X, but they don’t apply them automatically.

Nothing alienates a user more than being inconvenienced by their OS, and there are lots of free alternatives these days.