TPersistent.com

One of the great things IMHO about hcOPF is the design-time binding.  This feature requires you to install a design-time package into the IDE containing the MetaData for your objects.

The hcUIObjectBinder component uses the ThcUIBindingItems specified by the developer to create the appropriate mediator class at run-time and bind the specified attribute (by name) to the specified control (by name).  If you name your controls consistently it would even be possible to add automatic design-time binding by inferring the mediator from the type of control, and matching the name of the control and the attribute.  That feature isn’t available yet, but after configuring the bindings for 30+ controls on a form you can bet I have it planned.

My current project has grown to the point where the objects contain lots of code other than that used at design-time.  Initially I have used some conditional directives to limit the units implicitly imported into the object design-time package.  This has become increasingly difficult as things have grown and have added complications during the compile cycle.  If I’ve built the design-time package recently, then some of the DCUs are built with the conditional directives, so I either re-compile the entire main project, or change the Unit Output Directory for the design-time package to a separate directory (which is what I did).

To make it easier to keep fleshing out the objects, I have considered 2 different approaches:

1) Using Include files containing all the common code used by both the design-time and run-time objects.

2) Creating the base business objects for design-time use and concrete objects descending from them for run-time use.

Neither approach is ideal.

Include files break up the code so much it’s harder to navigate and make sure changes in one file don’t prevent compilation.  Too bad Delphi doesn’t support partial classes.  If it was just the object metadata it would be one thing, but the code for child lists has to be included so they can also be bound at design-time.  Using include files makes it easier to generate class definitions from the database, but I already have a utility that generates my entire unit so I have a good starting point for a business object.

Likewise, inheritance breaks up the class so reading it and understanding the whole run-time object can be more challenging.  This is typical of OOP design though.

What approach would you use and why?

Mike Rozlog just published a new roadmap for Delphi.  Apparently Delphi XE will be the big announcement at DelphiLive this year, not the preview 64 bit compiler,  or a public beta for Mac OS/X support as I had hoped.

Thew new roadmap mentions lots of Cloud Computing enhancements including Windows Azure and Amazon C2E.  This baffles me, since I know very few developers who are even interested in Cloud Computing, let alone actually writing code for it.  To me, Cloud Computing is still an experiment whose main participants are the big companies who can fund their efforts for a long time until they become profitable, or who make their money from advertising.  Cloud computing also rated 4th on “upcoming technology that would have the greatest impact on the development community at large”.  That was behind Mobile applications, 64 bit computing, and parallel programming.  I think it’s strange that such new technology is being incorporated into the Fulcrum release, when the survey didn’t ask when or if developers were going to target that environment.

What developers are targeting today, and tomorrow is the current Windows platform.  With Windows 7 copies flying off the shelves, you would think EMB would want to maintain it’s position as the best native Windows ADE.  Without 64 bit support it’s going to be difficult, and true 64 bit support will not be available until Project Pulsar is delivered (no timeline is given).  The preview compiler which will be available in 6+ months will not be able to generate Windows GUI applications if I interpret the slides correctly.

While the roadmap indicates that numerous projects are being worked on simultaneously, we’ve been hearing that since the last roadmap, and a year later the result is Delphi XE, now aka Fulcrum.  So far, from what I’ve seen, there is nothing earth shattering in this release.  In fact nothing worth upgrading for, and nothing that is a top priority request from the community or market.  To be fair, the top two items are big ones to deliver (64 bit Windows and Mac OS/X support), but could have been anticipated long ago if EMB had their finger on the market pulse. After a year of waiting it would be nice to at least see some progress, rather than Fulcrum features becoming what’s available in XE, and having everything else pushed back into Pulsar (now you see it now you don’t).

My concern is that resources are being consumed to deliver features to satisfy needs that have already been met, or are not true needs.  Subversion support is a great example of the former, and Cloud Computing the latter.  Why re-invent the wheel when most developers are using other IDE integrations, or Tortoise?  At one time this would have been a great addition, but that time has long since passed.  It still adds some value, but isn’t a priority.

I agree with Mason Wheelers’ latest blog post.  I certainly wasn’t impressed, but I was amused by the subtitle on the first slide “What we are telling customers we are going to go”.  Most excellent english.  It seems like EMB doesn’t understand that the community needs some assurance that a Pulsar release is imminent.  It’s interesting, and at the same time discouraging that EMB is repeating the mistakes Borland made.  Perhaps if you keep the same people, you keep the same company, and it’s just business as usual.

One of the things I like best about using an OPF is that an application can usually be sectioned into re-usable pieces if you aren’t dependant on the data layer chosen, and using an OPF inherently removes a great deal of your dependancy on that data layer.

Once you choose Oracle as your database, anything that talks to the database is no longer re-usable if the database is changed to SQL Server for instance.  You can of course, limit these dependancies by crafting the code with this in mind, and limiting any database stored procedures or functions utilized.  Unfortunately, it’s been my experience that this just doesn’t happen.  Either it’s development time constraints to remain competitive, lack of foresight, or just a poor choice for the initial RDBMS.  By the time the product gets to the point where the database needs to be changed to provide better scalability, or suit a target client, it essentially requires re-development because it’s too tightly coupled to the DAL.

Using a DAL like DBX, or ADO helps in that you can connect to a variety of different databases, but it doesn’t solve the problem that different databases support different column types and SQL syntax.  For example, Interbase doesn’t support the bit datatype commonly used in SQLServer.  To handle such differences you could create separate datamodules and try to abstract them so on start up the application could determine which datamodule to use.  This requires a great deal of effort and forethought, which in practice, seldom happens.  Conditional compilation is another option, but it can get quite cumbersome.

An OPF solves the datatype issues by providing translations from the Field types the selected Delphi DAL maps to the database columns, to object attributes.  A boolean object attribute can be stored as a bit in SQLServer and a Char(1) in Interbase, and all that has to be changed in your code is a single line that defines the attribute metadata.

You still have to deal with any database specific code if you choose to override list Load/ApplyUpdates or object Read/Write methods, but the RDBMS specific code is drastically reduced.  It works so well, that I used hcOPF to enable me to work on an Oracle 10G based application at home.  I didn’t want to install or maintain Oracle on my home machine, so I used Firebird at home and Oracle at work, with almost identical source code, using a conditional directive to isolate the minor differences.

The added benefit is that you can code re-usable application pieces, such as the classic Settings or Options dialog found in most applications, including Visual Studio.  Using dynamically created mediators, the entire dialog can be data driven, and re-used across projects regardless of the database employed.  The only requirement are the basic Settings table structures.  Supporting the editing and validation of a new setting is as simple as creating a database record.  This is one step above component development, and could even be deployed in a package to maximize re-use.  Don’t expect this code to make it into hcOPF though, it’s not my IP to give away.

I attended WWDC this year, in part because I knew there was going to be a new iPhone release and it would likely be accompanied by new a new version of the iPhone OS.  Of course there are also lots of tidbits regarding Mac OS/X and Apple’s PC hardware.

I also considered attending Delphi Live this year.  My current project timelines would have made that difficult, but I was prepared to justify it, except for one thing; there was no promise of any information regarding Fulcrum, the 64 bit compiler,  or a preview of either.

I’ve been developing with Delphi for a long time now, so most of the benefit for me is in seeing what’s new.  I’ve collected a lot of resource material over the years, and while I still learn something new all the time (at the expense of forgetting something else - let’s face it; the noggin is full), I value conferences mostly for getting me excited at the prospect of something new and potentially profitable.

I didn’t get the impression that the last Delphi Live was earth shattering, and when I emailed Frank Stepan to get some additional details on the Intellibook offer, because it wasn’t very clear, he informed me that “To receive an Intellibook notebook you are required to register for a minimum of 3 days”.  Suddenly, Delphi Live didn’t look so appealing.

If the organizers of the conference read this, might I suggest a couple of changes to their approach?

1) Always make sure there is as big an announcement as possible at each conference so attendees are the first to know, and if preview editions of upcoming releases are available, make it part of the conference goodies.

2) If you’re going to solicit early bird registrations, at least make sure the conference agendas are full.  When I looked there were a couple TBAs only 3 days prior to the end of early bird registrations (in fact I just checked and there are still 2 TBAs).  I want to know what I’m going to get for my time and money, and need to know so I can decide what days to attend.

3) Publicize any software discounts from third party vendors and exhibitors.

4) Schedule the conference around the same time as other complementary events.  I am thinking specifically of WWDC.  If Delphi Live is only a couple of days, it might make a lot of sense for it to be before WWDC or after.  If EMB is going to produce a development tool that targets Mac OS/X it would make a lot of sense to make it easy for such developers coming from afar to attend.

5) if an Expo is an integral part of the conference, shouldn’t there be at least a listing of Exhibitors?

Yes, you should definitely read with your children, but that’s not what this post is about.

Normally when a root object is read in from the object store, all it’s children are also read.  This is not always desirable from a performance standpoint, and it wasn’t an issue until I introduced complete design-time support for bindings.  Now, in order to bind lists, they must be created and added as children to the root object (I use an AfterConstruction override).  Previously, to get deferred loading, you would just create the list and add it as a child, just prior to loading and consuming the list.  While you can still do so, it’s not nearly as convenient, and it means your code won’t be as consistent.  Thus, I have introduced a new property on ThcObjectList called ReadWithChildren.  The default value is True, so the bahaviour will be the same as it always has, but now you can disable it whenever necessary.

In case you’re wondering when you might ever use it, I will provide one such scenario.  I have an Client object with  list of benefits that I have broken out into categories.  Since ThcObjectList doesn’t currently offer very much in terms of list filtering functionality, I have created two lists that I populate when I call a property getter to populate the complete list of benefits.  An added wrinkle is that benefits can expire,  so after I load the main benefit list, I check each benefit to see if it has expired, and if it has I mark it as such, and remove it from the list.  The remaining benefits are added to each separate category list.  Using three lists may seem like overkill, but they share the same objects, and it also makes it easy to change the contents of the grid in which I display them.  I use a single control, and simply change the list the mediator uses at run-time depending on the filter button that is down.  Here is the code fragment I am using:

  //unbind current objects
  Client.UnBind;

  //change package list displayed in grid
  GridMediator :=  (obClient.Bindings[1].Mediator as ThcTcxGridMediator);
  GridMediator.UnBind;
  if ToHealth then
    GridMediator.ObjectList := Client.HealthBenefits
  else
    GridMediator.ObjectList := Client.FringeBenefits;

  //re-bind to update UI
  obClient.BoundObject := Client;

If you have ever questioned the value of filling out the Embarcadero developer survey, let me assure you it’s well worth your effort.  It provides EMB with valuable feedback, and even if you don’t like what they do with the information, you can’t complain unless you participate. Kind of like voting for your “favourite” politician (but I digress).  If you do participate, there is the chance of an unexpected benefit.

I got an email recently from Deanna Dames at EMB.  Apparently, I won an Amazon Kindle for filling out the developer survey.  Just wanted to say thanks EMB!  Now I can recycle all those Delphi books and put their PDFs on my kindle.

A common problem that developers have to deal with is validation of input against a lookup table in the database.  Sometimes the “table” isn’t even in the database so a hard coded list of key/value pairs is used instead.  In each case, hcOPF provides a solution to eliminate all the boiler plate code you might use to populate a TComboBox or similar control with such values.

Static Values

If the Key/Value pairs are small in number and static then an array of ThcKeyValue pairs can be declared as such:

MaritalStatusKV :array[1..2] of ThcKeyValuePair = (('S','Single'),('M','Married'));

Then simply use the ThcComboBoxKVMediator in the binding for the TComboBox that displays the MaritalStatus on the form and in the FormCreate event, use the following class method call to populate the control:

ThcTComboBoxKVMediator.LoadItems(cbMaritalStatus.Items,MaritalStatusKV);

In the Database

If the Key/Value pairs you need to use reside in the database, its just as easy.  In a TDatamodule, drop on a ThcLookupList component.  Enter a SQL select statement that would give you the result set you want, and then enter the KeyField and ValueField.  In the form where you need to validate input, use a ThcTComboBoxLookupListMediator.  Choose the LookupList you dropped on the datamodule, the control and the attribute and you’re done.  The control will automatically be populated when the form is streamed in. Remember to choose the ForeignKey attribute for the attribute to bind since hcOPF automatically populates key information prior to saving objects.

The cool thing about this component is that it dynamically creates MetaData from the select statement you enter.  All the columns in the select statement are added to the ThcObjects used to populate the TStringList.Objects, so if you need to use more than a simple Key/Value pair in the object, it’s still a breeze.  For now, if the Key value is Null, the ThcComboBoxLookupListMediator automatically selects the first item in the list.  This may not be the desired behaviour, so a custom mediator may be required.

There still needs to be some design-time work done for this component, but it’s functional and available now in SVN if you want to check it out.  Only D7 projects have been updated with some of the latest changes.  I haven’t checked out how the JEDI deal with syncing multiple projects across Delphi versions yet…

One pending improvement in hcOPF is that mediators need to take on the responsibility to get the control to indicate in a consistent fashion that an attribute is required, read/write or readonly.  Currently this is handled using RTTI by the ThcUIObjectBinder when the object is bound.

Objects are bound only once when a form is shown, so each control is only ever updated once to reflect the properties of an attribute.  This can be a problem when you have an attribute whose value you generate a default for in new objects, but allow the user to change it, and for existing objects is ReadOnly.  The way to handle this currently in hcOPF is contrary to the way I’ve typically seen it handled in a Delphi application with non-data aware controls.  Normally the developer would interrogate the object or attribute for it’s status and change the UI control appropriately.

In hcOPF the framework does this for you, most of the time, but in this scenario it cannot.  The reason is that ThcAttributeDef.Properties contain the ReadOnly property and they were never designed to be changed dynamically.  The ThcAttributeDef has no knowledge of the attribute to which it’s attached to prevent circular references, and minimize memory usage.  All ThcAttribute instances created from the ThcAttributeDef have a pointer to the ThcAttributeDef so they can defer calls for information contained in the definition to that object, making it a many to one relationship.  I could have added a multicast event in a ThcAttributeDef.SetProperties method to notify all attributes using the ThcAttributeDef that a property change had occured, and then they could call their Modified method, but this would consume more memory and complicate things for essentially an edge case.

What’s the solution?

First of all, the ThcAttributeMediator you’re using must update the state of the control in the UpdateControl method.  None of the current mediators in the framework do so.  That’s about to change.

Then you need to update the ThcAttributeDef.Properties to include or exclude the apReadOnly property and force an update of the UI control attached to the attribute.  You can do this with a call to

hcObject.ObserverIntf.NotifyOfSubjectUpdate;

OR

UIObjectBinder.GetAttributeBinding(’SomeAttribute’).Mediator.UpdateControl;

OR

since Mediators references are stored in the Tag property of each UI control, this would also work:

(TObject(edSomeControl.Tag) as ThcAttributeMediator).UpdateControl;

The last two calls will be more efficient as it will only update a single control, where the former updates all controls attached to the subject.

In addition you will have to toggle the Enabled property of the control (if applicable) since the current implementation of ThcUIObjectBinder toggles it to False for ReadOnly attributes.

Expect some major changes in the ThcUIObjectBinder and mediators soon…

CJH made some valid points about my last article.

I guess I really wasn’t that clear on how my story showed the benefits of Open Source.  Let me attempt to rectify that.

Open Source can mean different things.  You can read most of the VCL source, and that has numerous benefits that could not be achieved otherwise.  The VCL source is a kind of restricted open source since the EULA restricts you from doing all that much with it.  It’s still very useful in that you can find out what method to override to achieve the desired functionality, or find a more efficient algorithm.  You’re not reliant on documentation that can be grossly incomplete or incorrect.  Nothing is better at teaching you how to code, than reading good code, and if it makes it into a commercial release it should be good right?  Well perhaps not as good as it could be, because how else would you ever be able to get over an order of magnitude of a performance gain.  So point #1 is that without the ability to read the RTL/VCL source an add-in like DelphiSpeedUp would have been more difficult to craft.

I have collected many Delphi books over the years, and some of them like the “Hidden Paths of Delphi 3″ by Ray Lischner really opened up the Open Tools API.  Information collected by experimentation, talking to Borland engineers, and no doubt reverse engineering that was shared within the community helped make the IDE a platform for innovation.  Out of that information came products like CodeRush and Castalia.  Open Source projects like GExperts, CN-Pack produced add-ins that provide functionality that should be part of the commercial offering.  So point #2 is that EMB should recognize the benefits that Open Source has brought to developers who use their product, and facilitate these projects/products so they can thrive, and integrate better with the IDE or toolset.  CodeGear started down this path by making FastMM4 the default memory manager, but there is so much more than could be done.  This might involve shipping with, and being part of the native Delphi install.  This might mean having their own open source hosting service for Delphi projects.  If it makes sense for RemObject or Gurock software why not EMB?  What about dedicated people to work on open source projects and incorporate the results into the VCL?

Point #3 is that  having the VCL source code likely prompted some of the Delphi developers to read it and say “we can do better”, leading to projects like FastMM4, FastCode and KOL.  Delphi Speedup leverages the FastCode project to do it’s magic.

It would be nice if EMB would open up more areas, and accept contributions directly from the community, but I don’t see how that is possible unless they entertain a free version.  Not too many people run Darwin on their Mac, and no one will contribute code for free and allow someone else to benefit financially from it without giving back.

That was the whole point I was trying to make (albeit indirectly).  I think EMB needs to open source the compiler leveraging LLVM (look at Apple’s XCode 4), standardize and publish the language specification for all to use, and open up the IDE so tools like Castalia, and GExperts don’t have to implement their own code parser.

Borland realized some of the benefits of Open Source by releasing most of the VCL source code.  It didn’t hurt their business. Even Firebird helped increase the number of developers using the technology, and when they wanted features like SMP support and wire encryption, they turned to Interbase.  Developers are a unique audience when it comes to Open Source.  We can actually contribute in a meaningful fashion, and EMB doesn’t have the resources or time they need to hit the necessary targets.  If EMB wants to see Delphi everywhere they are going to have to compete with Apple’s XCode 4, and provide a 64 bit compiler ASAP.  I think they can use all the help they can get, and that makes Open Source more appealing than ever.

I have always enjoyed the speed of Delphi’s compiler.  It definitely sets Delphi apart from other native toolsets like Visual C++.  Over the last several weeks work on my latest project was grinding to a halt because for some reason the Delphi 7 compiler was taking any where from 3 to 3m22s to compile my project.

I’m sure many people out there would simply say “upgrade” to solve the problem.  That’s where it gets a little complicated.  I had evaluated Delphi 2010, and Delphi 2007 early on during project development.  The existing product version is in fact written with Delphi 2007, but I chose to use Delphi 7 instead.  Why?  In one word, CodeRush.  CodeRush is a very old product, but provides templating and refactoring support that is still beyond what is offered in the native IDE.  I’m also very comfortable with it.  The problem with “upgrading” is of course all the components used.  I have source for them, but migrating all of that, including design-time code can be rather onerous.

Besides, as a programmer, you need to find out why.  My other projects including hcOPF compile lightning fast as always.  So I contemplated what made this specific project so unusual.  Currently it only contains 64 units, uses the DevExpress grid suite (of course), the JCL/JVCL, hcOPF and a smattering of other small components.  One of the major differences was using the JCL/JVCL, and I know that it sucks in quite a large number of units the way it’s organized, but I still didn’t think that could be the cause.

I finally decided to post on the embarcadero newsgroups.  The Delphi community is still pretty vibrant, and there are lots of folks out there who have been using the product from day 1.  I got several suggestions including:

Put only .dcu, .dfm, .res, .inc file in the LIBRARY PATH, and .pas files only in the Browsing Path.

This did seem to help, but compilations were still about 3 minutes.

Then Marius suggested I look at Delphi SpeedUp.  I had ran across it earlier in my googling for performance improvements, but never considered it a solution to my problem because my performance problem was project specific.   Was I ever wrong!  Compilation times went from over 3 minutes to 11 seconds.

So this is the second time I must thank Andreas Hausladen; once for updating the JCL/JVCL to use the LIBSUFFIX directive to make it easier to manage my hcOPF packages for different Delphi versions, and now for Delphi SpeedUp.  Thanks also to the contributors to the FastCode project.

This just goes to show what Open Source can do for commercial products.  I think Embarcadero could leverage Open Source more effectively to build out the VCL, and improve the IDE.  Even the compiler could benefit from Open Source projects like LLVM.  Apple seems to have prospered by leveraging Open Source…