TPersistent.com

I have finally started re-factoring hcOPF to support FireMonkey and Win64.  Win64 was a breeze, but supporting FMX is proving to be a bit of a challenge.

If I was EMB, I would be trying to make FireMonkey a write once compile on many platforms solution, and it is across Linux, Mac OS/X and Windows AFAIK, if you build an FMX application.  However, I would venture that most customers are looking to port their existing VCL codebase to access more markets, or at least leverage their existing knowledge, and might be a little hesitant to bet it all on a newly released UI framework sold and supported by a single vendor (especially after CLX).

All developers know, changing code tends to break what once worked, especially when you introduce more conditional compilation (check out my QC request 94287 to make conditional compilation easier to use).  To accomplish this end, the API usable to FireMonkey applications needs to have as much in common as possible with even VCL for Windows applications because at some point a developer will use code to manipulate the controls.  If it’s possible to use the same code for both platforms, you’ve saved in not having to maintain functionally duplicate code and in dealing with conditional compilation.

The ideal scenario would be to be able to specify either a FireMonkey or a VCL form DFM in a conditional directive, and have all the UI code shared (or minimal conditional compilation).  Of course, if you want to use the advanced functionality available in FireMonkey, perhaps this approach isn’t viable.  If you just want to target Mac OS/X without having to re-write your VCL application forms, and are using FMX for basic presentation of data, this would be ideal!  You wouldn’t have to invest the same effort to determine the merits of FireMonkey as a UI replacement for the VCL because you wouldn’t need to keep two separate UI source code trees in sync while FireMonkey matures.

So far, with my brief exposure to FireMonkey with XE2 I can see a number of problems in achieving the Write Once Compile for Many Platforms concept.  FireMonkey uses .Text instead of .Caption for some control window captions (ie: TGroupBox).  So while the control class may be the same, even some simple UI code cannot be shared with a VCL application.

Even non-visual code may be a challenge to re-use.  For instance, I use GetTickCount() to time activity in hcOPF.  I want to keep hcOPF compilable for users of D7 and above.  I personally feel that my coding productivity was higher in D7 with CodeRush than the Delphi XE with GExperts and CnPack.  Part of the reason for this is the code parsing the IDE performs in the main thread.  Type in Begin incorrectly, and you can be waiting for 10 seconds while the IDE tries to figure out what is going on.  That’s pretty sad on a 6 core system…but I digress.

GetTickCount() is implemented as an inline function in the implementation section in the System unit for MacOS and Linux, and the Windows implementation is in Windows.WinAPI.  Not a big deal to add a few {$ifdefs} to handle that, but GetTickCount() as defined in System is not accessible to other units.  So in order to use it, you will have to copy the implementation from System, and expose it.

From what I have seen of XE2 Update 4, FireMonkey is still not usable from a performance standpoint for replacing a normal VCL form. On my 6 core system with a Radeon 4250 the main form appears and you can visibly see the contents of the form background and content being rendered.  In this case, the form is just 4 edit controls in 2 separate group boxes using hcOPF Object Binding to populate the controls.

I think FireMonkey is a great concept, but adoption would be faster with an API more consistent with the VCL, and I think FMX has a long way to go before it becomes a viable replacement for the VCL and X platform Delphi becomes a true reality for more than the most trivial application written from the ground up for FireMonkey.

When using the DevExpress grid with ObjectLists, there is currently a limitation that you must assign a root object to the hcUIObjectBinder.BoundObject property and this root object needs to expose the ThcParentedObjectList to present in the grid as a child of the root object that exists when the root object is constructed at design-time.  It may be empty, but it must be present, and provide a ListName in order for the object inspector to provide it as a discernible option.  Hopefully sometime in the near future I will have time to make it possible to bind to a standalone object list.  Here is a code snippet that I use in the AfterConstruction override:

  FLoggedInUsers := TftEmployeeList.Create(Self);
  FLoggedInUsers.Name := 'LoggedInUsers';
  AddChild(FLoggedInUsers);

While the ThcObjectDataSource does the majority of the heavy lifting for you in terms of presenting objectlists in the Developer Express Quantum Grid, there are times in which you need to programmatically modify the domain objects and inform the grid to update itself.  This can be achieved by calling:

GridTableView.DataController.CustomDatasource.DataChanged;

or the DataChanged() method of the TcxGridMediator.

In order to ensure the domain objects are updated immediately, I always set the ImmediatePost property of all grid editors to True.

Another thing to look out for are changes that are pending in the grid that have not been committed to the underlying objects.  For this I use the following code on the TAction.OnExecute event handler in my forms:

  if GridTableView.DataController.IsEditing then
    GridTableView.DataController.Post(True);

Of course you still have to persist the data if you’re using ThcParentedObjectLists that use CachedUpdates.  By default lists do not use CachedUpdates, so the grid Posts the data when you scroll to a different row, and the object layer writes the object data to the database.  The code above also results in a database update for for the current object in such lists.

If you use these tips, using hcOPF objects with the DevExpress Quantum Grid can actually be easier in some cases than a dataset.

If you assign the Min and Max Value properties of a TcxEdit descendant, the control will call the OnValidate event handler indicating there is no error, but a subsequent attempt to convert the displayvalue into the target datatype and check if the result is within the range specified will result in an exception being raised. If you’re using something like Eurekalog or MadExcept to trap all unhandled exceptions this can be a major pain in the derrier.

There are only two ways I’ve found to deal with this issue:

1) Tell MadExcept or Eurekalog not to process this exception type

2) Do not define the Min and Max values in the control. Instead let the hcOPF validator deal with this, and I populate the OnValidate event with the following code to prevent an earlier exception from being raised (this code is for a cxDateTimeEdit):

if Error then
begin
  MessageDlg('The Date Entered is not Valid!',mtWarning,[mbOk],0);
  DisplayValue := DateTimeToStr(Now);  //reset the display to a valid value
  dtpBirthDate.SetFocus;
  Error := False;  //don't raise an exception since we've handled the error
end;

I just installed the newly released XE2 and attempted to compile my current client’s project in the new IDE so I could start using it on a daily basis.

One of the first things I needed to do was compile hcOPF for XE2.  I expected not to have any issues since I am currently using hcOPF with Delphi XE.  Much to my surprise, the compiler kept complaining that it could not find Windows.dcu - an odd error considering Delphi has been a Windows development tool for over 15 years but understandable once you know that Unit Scope Names were introduced in this release to accommodate X platform development and units were reorganized.

Unfortunately, unit names in Uses clauses have to be fully qualified (winapi.Windows in this case) and this means using a whole bunch of {$ifdefs} for XE2 to provide fully qualified unit names.  It’s certainly an ugly solution, but I am happy to say that hcOPF now compiles under XE2 with the exception of the DevExpress package which must wait for a vendor code update.  The latest changes are in the SVN repo.

In more complex application scenarios, especially multi-threaded ones, it is difficult to determine when an object can be safely destroyed. The last thing you want to do is prematurely free an object that is referenced elsewhere and then try to track down all the mysterious AVs that only result in specific situations that users cannot describe sufficiently for you to isolate the problem.

For example, if you have a customer object they may be referenced by an invoice being processed, and a form to change their contact information. When the invoice is completed, the developer cannot simply assume they can destroy the customer object. There are several possible solutions to such a problem:

1) clone the object whenever it is referenced. This isn’t really a good idea. It chews up more resources, and data modifications on the clones can cause last-write-wins problems with valid data being over written with stale data. One of the benefits of an OPF is requiring fewer round trips to the database to refresh data than a TDataSet approach, but this only works if all changes are made through a single instance of the object.

2) Use reference counted interfaces. This solution has a couple drawbacks that I can think of.  You cannot use indexed properties in an interface definition. For hcOPF that would mean generating individual getter and setter methods for each attribute, rather than using a generic property definition such as:

 property Name :ThcAttribute Index 12 read GetAttribute;

That would require a lot more code generation and a much more brittle object definition that’s difficult to maintain over time.  In addition, hcOPF was designed using a Class model so the framework handles the object internally as class instances rather than interface references.  This would be a significant architecture change, and I don’t think it would be worth it.

3) Use manual reference counting.  With this approach there is minimal overhead; less than even interface ref counting, but the downside is it’s completely up to the developer to implement, and it requires keeping a list of the root objects that need to be destroyed.

The latest commits for hcOPF provide additional support for monitoring the RefCount of an object, adjusting it (Retain/Release) and logging it’s destruction with CodeSite.  Unfortunately, if you’re familiar with Retain/Release you might be hoping for something like Apple’s ARC functionality but it’s just not possible since it’s implemented as part of the compiler.

An oversight on my part when I wrote the validation framework within hcOPF was that users may enter values that exceed the range of the underlying datatype used by an hcOPF attribute. I assumed that the UI controls used would ensure that only values within the datatypes range would be assigned to the attribute.

It turns out this was a rather rash assumption considering the level of validation that some of the stock VCL controls provide. Consider the TEdit control for instance. In Delphi 7 it cannot even constrain the values input to only numbers. You have to choose a different control, namely the TMaskEdit, or a TSpinEdit to accomplish that. There are also many bugs in such controls that allow the user to input a value that should be considered invalid. The TSpinEdit, for instance, has a Max and Min Value which is utilized to validate the input when using the spinner buttons, but not when keying in the value directly.

In order to adhere to the DRY principle, and ensure that Range Check errors don’t occur for numeric attributes I had to modify the way the framework handles values from controls. Normally, hcOPF mediators would assign the value of the control to the attribute, and the attribute would be checked to see if it, and the resulting object as a whole was valid. This was to facilitate error indicators, and consistent validation message handling without throwing exceptions that products like Eurekalog would trap and report as errors.

So validation has now been re-vamped such that the Attribute performs a range check before attempting to assign the value in it’s native setter (the one matching it’s datatype). Instead of throwing an exception like a TField does, the attribute ignores the new value and sets an error message so the error indicator framework can reflect the condition in the UI. If you don’t use the error indicator framework, you would have to test if the lastValidationMessage is not empty and display it in a message dialog, and then refocus the control.

I just updated the hcOPF package generator.  This project is a utility I started work on in order to simplify the maintenance of hcOPF packages.  It’s far to time consuming to update the packages for all Delphi compilers supported every time I make a change to the one I’m working on.  If they get out of sync, it becomes a nightmare for someone unfamiliar with the framework to resolve.  I looked at the Jedi package generator, but it’s a complex beast with support for C++ Builder, which I don’t need.

I also considered building an installer based on the MustangPeak component installer, but that’s on hold for the moment as I simply don’t have enough hours in the day.

My KISS solution was to chop up the DPK into three files suffixed  by what they contain (contains, requires, and directives).  The PackageGenerator application uses only native delphi controls so you should be able to compile it with any Delphi version you want.

The UI is pretty simple.

The hcOPF Package Generator

You select one or more compilers for which you want to generate current packages.  You must set the root compiler folder path so the application can find the subdir for each compiler version supported in which to save the generated files.  The default is “Z:\hcOPF\Compilers”.  I recommend using a short path and creating an environment variable in Delphi to point to the folder off of the root of the drive.  My environment variables are as follows:

hcOPF = $(HD)\hcOPF
HD = Z:

This way, I can change the drive mapping, or the root folder of the hcOPF framework and I don’t have to change any of my Library Paths  (the hcOPF packages also use the $(hcOPF) environment variable so defining $(hcOPF) is mandatory):

$(hcOPF)\Compilers\D7
$(hcOPF)\Lib\D7
$(hcOPF)\3rdParty\TPerlRegEx
$(hcOPF)\Source\Include
$(hcOPF)\Source\Resources

The following paths should be added to your Browsing Path:

$(hcOPF)\Source\Core
$(hcOPF)\Source\Common
$(hcOPF)\Source\DAL
$(hcOPF)\Source\GUI
$(hcOPF)\Source\Validators
$(hcOPF)\Source\Utils
$(hcOPF)\Source\Reporting

This means trying hcOPF should be as simple as:

1) Configure your delphi environment as described above.  That means creating an hcOPF environment variable, and populating the Library and Browing Paths as described.

2) Get the source from SVN.  Here are some instructions on how to use the SVN client to get the source.

3) Run the PackageGenerator - an EXE is in the Bin folder even though everyone should be able to compile the source.

4) Install the TPerlRegEx component.  The hcOPFValidators package requires this component for RegEx validation.  I have defined an interface so it’s possible to use your own RegEx evaluator if you wish.

5) Select the compiler you want to generate current packages for, and click the Generate button.

6) Open the AllPackages.bpg file in the target compiler directory for the compiler you selected.  For instance, if you checked D11 (Delphi 2007) then open the AllPackages.bpg file in \trunk\Compilers\D11.  In this case, Delphi uses MSBuild so it will upgrade the packages, creating the necessary XML files.  That is why the Clean Target Directories option is checked by default.

7) Select the first package in the group (HengenOPFCore.dpk) and Rt. Click - select Build All from Here

Rt. Click on each package from top to bottom that starts with ‘dcl’ and select Install

You should now have hcOPF installed and be able to compile the Demos (use the Delphi 7 projects as I have not yet updated the projects for other compilers.  I might actually adapt the PackageGenerator code to do that for me).

I have tested the output of the package generator with Delphi 2007 since I recently migrated my current project so I can compile under either Delphi 7 (yes I still use it as my primary IDE), and Delphi 2007.  If you have any issues, comments, or suggestions, let me know.

I did recently discover that versions of Delphi that use MSBuild do not preserve the DCU Output folder when upgrading the project.  Be sure to change the folder to ‘..\..\Lib\D??’ where  ?? represents the Delphi version.  I may change the package generator to generate DPROJ files.

A couple years back I was asked to develop an application for managing facility construction for an O&G firm.  The previous incarnation of the application was written using custom domain objects against an Oracle 8 RDBMS.  I agreed with the approach in using domain objects, but decided I would make my life easier and use an earlier version of what was to become hcOPF so I didn’t have to write the persistence logic and could use object binding.

Things proceeded well, but the SME (Subject Matter Expert) kept asking for more, and I ended up managing a pretty large object graph within the application.  It was then that I got a mysterious error message saying I had exceeded the query limit.  Me with my little application?  It seemed ludicrous that Oracle couldn’t handle a few queries, until the DBA told me it was 100 simultaneous queries per connection.  At first I couldn’t believe that I was using so many, until I did some investigation.  At that time I was creating a query whenever I needed to load a list or a single object, so it didn’t take long with an object graph of any depth to use that many queries.  I quickly implemented ThcQueryPool, and pretended that incident never happened.

That is, until quite recently.  On my current project, I needed to support Stored Procs and executing queries directly, and wanted to do so in a database independent manner.  That is, after all, one of the advantages of using an OPF.  I implemented this support in the form of interfaces that could be implemented by the TQuery and TStoredProc type components in each supported DAL (data access layer).  To speed up implementation, I created a couple methods on the ThcFactoryPool to return either a IhcStoredProc or IhcQuery.  I assumed this interface would be reference counted, and the components would be destroyed at the end of the routine in which they were used.  I was wrong, and just discovered this when SQL Server gave me a similar kind of message.  All I can say is “OOPS, I did it again!”.

There is a good explanation of this issue on StackOverflow.

In order to fix this situation I have implemented _AddRef, and _Release methods in the ADO DAL objects so reference counting is now used.  I will either have to propagate this solution to the other DALs, or re-factor to enable the use of Query and StoredProcedure object pools.

A while back, I came across an edge case with hcOPF that I had not anticipated when I wrote the binding mechanism.  When an object has calculated attributes, it forces the attached mediators to update the UI by triggering an ObjectChanged event which calls:

//notify all mediators of the change
FSubjectDelegate.NotifyOfSubjectUpdate;

which cycles through all mediators forcing them to update their controls.

Normally this isn’t a problem, but if you have outside stimulus that triggers a calculation event, such as a timer, or I/O device, then an update may be triggered while editing the object.  What’s worse, is that if the user is changing an attribute value, and the calculation is triggered before the editor control updates the object attribute (this will happen if the control doesn’t use an OnChange event), then the user’s changes will be overwritten.  It makes for a challenging situation for the user to make a change before the object is changed by the external cause.

What would be more appropriate is that the ThcCalcObject should trigger the UI updates for only the attributes it modifies.  This blog post should serve as a good reminder to both make this enhancement, and continue to use the work that you produce so you can see it through the eyes of a consumer (dogfooding).

One of the great things IMHO about hcOPF is the design-time binding.  This feature requires you to install a design-time package into the IDE containing the MetaData for your objects.

The hcUIObjectBinder component uses the ThcUIBindingItems specified by the developer to create the appropriate mediator class at run-time and bind the specified attribute (by name) to the specified control (by name).  If you name your controls consistently it would even be possible to add automatic design-time binding by inferring the mediator from the type of control, and matching the name of the control and the attribute.  That feature isn’t available yet, but after configuring the bindings for 30+ controls on a form you can bet I have it planned.

My current project has grown to the point where the objects contain lots of code other than that used at design-time.  Initially I have used some conditional directives to limit the units implicitly imported into the object design-time package.  This has become increasingly difficult as things have grown and have added complications during the compile cycle.  If I’ve built the design-time package recently, then some of the DCUs are built with the conditional directives, so I either re-compile the entire main project, or change the Unit Output Directory for the design-time package to a separate directory (which is what I did).

To make it easier to keep fleshing out the objects, I have considered 2 different approaches:

1) Using Include files containing all the common code used by both the design-time and run-time objects.

2) Creating the base business objects for design-time use and concrete objects descending from them for run-time use.

Neither approach is ideal.

Include files break up the code so much it’s harder to navigate and make sure changes in one file don’t prevent compilation.  Too bad Delphi doesn’t support partial classes.  If it was just the object metadata it would be one thing, but the code for child lists has to be included so they can also be bound at design-time.  Using include files makes it easier to generate class definitions from the database, but I already have a utility that generates my entire unit so I have a good starting point for a business object.

Likewise, inheritance breaks up the class so reading it and understanding the whole run-time object can be more challenging.  This is typical of OOP design though.

What approach would you use and why?