TPersistent.com

Tonight I was trying to install some hcOPF objects into the D7 IDE so I could use the design-time functionality to bind UI controls to the attributes of an object.  I created a package, included all units I thought were needed and then added all implicitly imported units or the packages that contained them.  I then tried a Build and got an ‘Internal Error U1295′ at the line where I include hcDevExpress in my Implementation section.

I thought I would Google the error, and I came up with a measly 4 hits with no answers, only others asking questions, although many of the pages had to be translated by Google.  I did stumble upon an article from Steve Trefethen here.  Oddly enough, searching EDN didn’t find anything on U1295, and searching on “internal errors” brought back results for multiple products.  It seems weird that there is no product filter in the advanced search options.  Furthermore, the Additional Compiler Messages window, which I didn’t even know existed after all these years, until I read Steve’s article, gave this link which is pretty much the article Steve published (nice circular reference).

So here I am, a Delphi developer for many years pulling my hair out (what little is now left) because I have to figure out what is going on by trial and error.  I can only imagine how souring an experience this would be for a Delphi newcomer.  You would think after 15 years a product might have more comprehensive documentation, especially on it’s support site.  Perhaps a more descriptive error message might have be in order, and then additional documentation would not be necessary.  Granted I am not using using the latest Delphi release, mainly because I am performing a large scale refactoring and I’m still a CodeRush addict, and this project has a single datamodule with a large number of queries which loads much faster in D7 than D2010.

Based on Steve’s article and my own experience, I re-compiled all my hcOPF IDE packages, and after the IDE died once, and I had to kill the process, I finally got the business object package installed.  As I suspected, I am using conditional compilation to include CodeSite messages etc, and if I compile a project that does not use packages with different conditional directives or compiler options than used to compile the packages, I often get into this kind of situation (not always the same Internal Error).  It makes for sporadically painful development experience, so I would welcome any suggestions from package developers who dog food their product as it evolves.

Ok I admit it.  I’m a bit of a hoarder.  I finally went through all my old paperwork and shredded everything that I didn’t need to keep for the CRA (Canada Revenue Agency).  Amidst 15 years of paperwork I came across the following:

Delphi 2 - The First 32 bit Windows Release

I ordered and returned Delphi 1.  It was an impressive product, but it was so new no one was using it, so it was a little hard to justify the money (I was even poorer then).  When Delphi 2 shipped, Delphi became my development tool of choice and has been ever since.

Embarcadero’s current pricing at the on-line Canadian store is $922.64.  I wonder how they determined that price.  It’s not rounded to the nearest dollar, or priced so it appears below some psychological threshold.  Even after GST is applied, it’s an odd figure.

Anyway, I wondered how much Delphi 2 would have been sold for in today’s dollars, so I used 3% inflation annually, compounded on a yearly basis and it worked out to $635.29 CAD.  The current price is 1.45 times the adjusted cost of Delphi 2.

There are certainly more features in the box, but do you think it’s worth the price?

I used to have some design-time support for the principle objects in the framework, but in my experience component and property editors are non-trivial to write, and the framework was evolving so much that I was constantly breaking all the design-time code.  I decided to back burner design-time support until hcOPF was more mature.  I am happy to announce that ObjectBinding design-time support is now available.

You can now design your objects, install the objects into the IDE, and bind their attributes to form controls without writing a single line of code.  Here is a screenshot of the ObjectListMgr demo showing the design-time support.

Many thanks to Developer Express, not only for donating a license to their DevExpress QuantumGrid Suite so I can continue to support it with hcOPF, but for allowing Richard Morris to share their technique for delegating properties to child objects.  You can find that article here.

The JCL and JVCL are a humungous collection of useful components and routines for Delphi.  Kudos to the team that maintains and develops them.  I have used the JCL/JVCL on a number of projects now, and my only complaint is that you can’t really pick and choose what components you want to use.

This came to light once again when I decided I wanted to use the TJvErrorIndicator component with the hcOPF.  I used a single unit from the JVCL and the next thing I knew, I had 3 more packages to distribute; the JCL, JVclCore and JvValidators.

One of the things I like most about Delphi compared to .NET is the ability to create an application with a minimal footprint.  Since I have open sourced hcOPF, I also don’t want to force anyone to adopt any components they don’t wish to use, or deploy anything larger than necessary.

I have looked through the dependancies, and there is really no need for most of the classes I am forced to deploy, they just happen to be in the same unit as something else.  I really wish the Jedi would adopt the SRP (Single Responsibility Principle) for units so I could pick and choose.  As a result, I am thinking about going over to the dark side and ripping out the ErrorIndicator code to create my own unit for hcOPF.  Not exactly an ideal scenario as I cannot contribute back any changes as easily, nor can I benefit as easily from any future enhancements the Jedi make.

Another annoyance is that for some reason the LIBSUFFIX directive is not currently being used for the Jedi packages from D7 up.  As a result I have to change the required package dependancy list for each project for each version of Delphi I am trying to support, rather than simply referencing, for example, JvCore.dcp I have to reference JvCoreD7, or JvCoreD14 etc.  I have reported this issue, and am assured it will be fixed in a future release.

Like other developers I often look to the community for guidelines on naming, code organization, and programming techniques.  What ever happened to dclPackageName${LIBSUFFIX} for design-time packages and PackageName${LIBSUFFIX} for run-time?  What is the recommended standard now?

Recently I have been enhancing the built-in validation capabilities of hcOPF in preparation for using it in a client project that requires visual error indicators, and flexible data entry.

I have introduced the concept of a Validator which is much like a database check constraint within the OPF layer, but it takes the concept much further.  The IsValid() method appears at both the object level and attribute definition level.  To enable support for any conceivable validation requirement, developers can override the default implementation on ThcObject, and attach custom validators to a ThcAttributeDef instance.

Validators are employed by the Mediators in the framework to take the string value from their associated control and after ensuring that it can be converted to the datatype of the associated attribute without a range error, their IsValid() method is called.  The prototype for IsValid at both the Object and Attribute level is as follows:

function IsValid(ValidationErrors :ThcValidationErrorList = nil):boolean;

IsValid() takes a list which is created by the developer and is used by the framework to collect any validation errors and their associated attributes.  The list can then be used to display the error information to the user, and controls containing invalid attribute values can be focused using the new UIObjectBinder.FocusControlForAttribute() method.

IsValid() has a default parameter of Nil so you aren’t obligated to pass it an instance of a ThcValidationErrorList.  This may seem strange, but it’s for a very good reason.  If you’re like me, and think that one of the best features of Delphi are TActions, then you don’t have to suffer the overhead of repeatedly clearing and populating the validation error list whenever a TAction.OnUpdate event handler executes.

For an example of how to use ThcValidators check out the ObjectListMgr project in the Demos/ObjectListMgr folder.

I’ve had a request for quite some time to provide an example of an eager load using hcOPF.  Quite frankly it was never a high priority for me, because I could never see myself using it.  That said, I like to think that hcOPF is a sufficiently flexible framework that it makes it easy for developers to do what they normally require, but still makes it possible for them to do unusual things.

One of the reasons why I like using an OPF (hcOPF in particular) is that it removes the tedious job of writing SQL CRUD (Create, Read, Update & Delete) operations.  hcOPF currently generates all the SQL for CRUD operations dynamically using the MetaData the developer defines.  I never worry about how to persist an object to the database any more, and to not use this functionality kind of defeats the purpose of using the framework.  Nevertheless, I recognize that there are times in which a developer would rather call a stored procedure to load and save objects rather than leave the SQL generation up to the framework.  Perhaps for security reasons, or in order to leverage some RDBMS feature such as MARS.

In any event, I decided to take up the challenge and write a unit test to demonstrate how to use a single SQL statement to load a result set and construct multiple objects from it.  I will leave saving objects using stored procs for a future article.  The code is located in the hcEagerLoad unit in the UnitTests folder.

The test is a descendant of ThcDBTestCase, which is based on the database test classes authored by Giacomo Degliesposti, who presented his test classes at DataRage and was kind enough to send me a copy of his source.  Many thanks, as it helped get me started writing tests against a database which although slower than mocking it, is pretty much a necessity for a database centric framework.

ThcDBTestCase handles all the connectivity to the database for hcOPF and provides some handy methods for running database scripts.  Any descendant just has to create any database records they will use for testing, and then create the corresponding objects in the Setup method.  After crafting the actual test, the TearDown method simply has to free any objects created in the SetUp method.

Here, the actual TestEagerLoad method uses a single SQL statement to fetch all People with any associated Addresses.  Using two nested loops, it Inserts new Person and Address objects into their respective lists, populating the object attributes from the query.  Then the code informs hcOPF that the objects actually exist in the database using the ThcObject.ObjectExistsInStore method.  This resets all the original values populated from the database when the object is initially read, with the current values, so the framework will build the correct SQL and populate the parameters properly for any modifications made to the objects from that point forward.

Note how hcOPF uses a similar construct to TFields in being able to access each attribute by it’s name (AttributeByName() method), but that it’s also possible to define the objects with indexed properties that access the underlying attributes directly using type specific getters and setters, or by using custom getters and setters.  Either way, you can write anAddress.City or Person.LastName with full code insight.

Every new client project that I work on, just underlines the benefits of an OPF to me.  Less SQL executed against the database, no business logic scattered throughout forms, and most importantly the ability to withstand change without the same kind of breakage (especially if you write unit tests).  That’s just MHO, what’s yours?

Thanks to Bo Sorensen, hcOPF nows supports the latest Delphi compilers using Unicode as the default string type.

I upgraded to RAD Studio 2007 a while back as a show of my faith in CodeGear, and to put my money behind the future of Delphi.  Although I own a license, I never really used it that much until recently.  Many of my projects remain in Delphi 7, but as an old time user I thought it was about time to learn some of the new features and functionality in a more recent IDE and reap some of the benefits of that investment.

IMHO BDS releases before D2007 were far too slow and used to much memory, but most importantly they weren’t very stable.  So far, while the D2007 IDE is stable and seems almost as fast as Delphi 7 (this is one of the primary reasons I bought the upgrade),  I still I have a number of issues:

1) Compilation cycles are longer.  Most likely this is due to MSBuild integration, but anything that slows down the compile/debug cycle is detrimental to developer productivity.  I don’t think making a single change to one line of code in one unit and re-compiling should take 20 seconds, even if it’s in a VM on a 2.5 Ghz MacBook Pro.

2) The IDE pauses for extended periods as it re-parses the code, while I am trying to type.  When a re-parse gets triggered an hour glass appears and I’m dead in the water until it’s done.   This is apparently addressed in D2010, but it really makes you wonder if they broke it sometime after D7 so they could fix it later and get developers to “upgrade”.  I’ve never had this happen in Visual Studio, and wouldn’t expect any professional development tool to impede your coding ability.

3) Code navigation

If you don’t practice one class per unit, you definitely need some way of navigating between classes and different sections in the unit.  I loved CodeRush’s navigation editor toolbar.  GExperts is now my tool of choice since it’s free, works well and is still supported.  I tried an older version of Castalia since it seemed to have a lot of the features of CodeRush, but it didn’t work as intuitively IMHO and it wasn’t as stable as I hoped.  What GExperts is missing is a way to jump to different sections in a unit, like the uses clause in either the implementation or interface, or an object definition etc.  The uses clause manager (Shift+Alt_U) covers some, but not all of this need.

While the structure pane has been around for eons, and has been further enhanced in D2010, I don’t use it for code navigation.  I guess I got used to CodeRush’s navbar, and the structure pane has additional, distracting information like units used, fields etc.  I suppose you can turn them off but if you have a method like Dummy() on two classes, searching finds the first one, but AFAIK there is no way of locating subsequent ones directly using the keyboard.  It’s also amazing to me that it’s called the Structure Pane, yet if you Rt. Click and select Properties on the local menu, the property dialog is titled Explorer Options.  Nothing like consistent names in UI design.

You would think that advanced code navigation would be one of the primary reasons for buying an IDE, but apparently it’s not since the concepts embodied by now defunct commercial tools like CodeRush, and open source efforts like GExperts still have not found their way into the IDE.  Those that have, seem to be more limited in capability, and you have to ask why…

4) Templates and other productivity features

CodeRush has some pretty good template support with the ability to drop markers and jump to them at a later point.  It’s a snap to define your own, and IMHO it’s still far better than anything available in Delphi today even though there are some bugs that affect insertion mode on occasion, and it’s parsing is not always 100%.

When was the last time you looked into writing your own live template?  Could you even find good documentation?  How do you tie in a scripting engine and access the editor buffer so you can manipulate the code?  It might be easier if Embarcadero exposed the Kibitz parser through the OpenTools API.  Right now, any IDE Add-in has to parse the code itself, and that leads to support issues as Embarcadero evolves the language.

Mark Miller left the Delphi world and created CodeRush for Visual Studio, and there is also of course ReSharper.  I have used both, and I must say that RAD Studio still lags behind the intuitiveness, and intelligence of these IDE add-ins.  While I understand that Embarcadero doesn’t want to put any partners out of business, there aren’t that many players in the market any more, and the lack of smart productivity features isn’t doing Delphi any favors.

Some of the features like declaring a variable using Ctrl+Shift+V are available, but unlike CodeRush which coincidentally uses the same key binding, it will not allow you to define a Field variable only a local.

Ctrl+Shift+C can be used to declare a method prototype if you have created it’s body in the implementation section, or the reverse (which I have seen used more often).  Unfortunately, in the former scenario it places the method prototype in the private section, not providing an opportunity to specify the method’s scope, or any additional directives such as overload, override, reintroduce etc.  CodeRush had this back in D7 days.

Error Insight is nice, but not as useful as it could be if you don’t know where all the errors are.  Resharper shows all potential problems adjacent to the scrollbar so you can navigate to the code by clicking on the colored indicator, and correct the error before re-compiling.  When compilations take longer, it would be nice to locate and fix as many errors as possible without having to re-compile to find them.

5) Refactoring

Much of the refactoring support relies on enabling modeling.  Even then, it doesn’t always seem to work, and I can write better documentation on how to use it, than what is supplied.  I attempted to rename some variables across units with Ctrl+Shift+E, and after I clicked on Ok, the code didn’t change, nor did the preview of changes dialog appear.

One of the features of CodeRush that I used the most was it’s ability to surround a selected block of code with a try/except or try/finally.  I tried D2007’s Surround tryf and it does inject the code, but it never indents it properly.  Maybe thats why they incorporated a code formatter in D2010 ;>)

Sync edit is a great concept, but it requires you to highlight the code block first.  Most of the time, you either want to rename a method variable, or a field variable so I don’t understand why it cannot determine the scope for you based on the identifier you’re changing rather than requiring you to highlight the code, invoke sync edit and then change what you want.  I wonder if Embarcadero has someone who actually looks at usability.  Hopefully this will improve as they increase use of Delphi internally.

6) Advanced Search

CodeRush and GExperts both contain a much better GREP/Search across units than the native IDE.  This appears to have been addressed in D2010.

7) Documentation generation

One of the reasons I bought D2007 is because I read the compiler would now generate XML documentation.  It may generate the raw information, but it’s a long ways from real documentation so I’m still using external documentation tools which must again, parse the code themselves.

Conclusion

While I am still learning what D2007 has to offer, I am compiling my list of “must haves” for me to upgrade to D2010.  For now, even with the BOGO offer and all the great new things in D2010, I’m not putting out $620 CAD until I see that D2010 has addressed the issues above.  Time to checkout the CodeRage replays…

What would make you upgrade?

I have been looking for a tool to generate documentation from embedded comments in hcOPF.  I am more familiar with JavaDoc than XMLDoc conventions, but I really don’t care which one I end up using as long as the tool can accurately parse Delphi code from D4 on up.

With the introduction of generics and class helpers in the latest versions of Delphi finding a tool that can parse Delphi source is not a trivial concern.  Although hcOPF doesn’t currently use any language features that would exclude earlier Delphi versions, it may at some point.  Even if it’s done using conditional directives, this may break the documentation parser, so selecting a robust parser that is going to be maintained in the future is important.

Needless to say, hcOPF is an open source offering so I’m not about to drop big $ on a tool to provide documentation on the framework.

I even went so far as to purchase Rad Studio 2007 because the feature matrix indicated it had documentation generation built into the compiler.  Unfortunately, the documentation on the parser and live templates that would be used to support the commenting efforts, is practically non-existent.  Since not everyone who will choose to use hcOPF will be using BDS2006 or higher, they won’t have the ability to use this documentation system, so I am leaning towards an independent toolset.

What do you use, and why?

hcOPF uses TField like attributes to represent the attributes of an object.  These ThcAttribute descendants provide an object with the ability to Undo any changes made to it’s attributes and notify other Delphi objects when changes have been made.  They are the foundation of the framework, providing a generic, re-usable way of representing object attributes without using simple types and Reflection with it’s limitations and overhead.  In order to specify the attributes for an object and how they’re stored and used, the developer needs to provide the framework with information in some form.  This is Metadata, and although it could be represented in various ways, including XML, in hcOPF it’s currently code.

MetaData is used by the framework to construct an object with the appropriate attributes, based on how the object is stored in an RDBMS.  Every ThcObject descendant must override the Register class method and register it’s metadata with the framework.  A typical Register method would be as follows:

class procedure TPerson.Register;
var
  MetaData: ThcMetaData;
  aDef :ThcAttributeDef;
begin
  MetaData := ThcMetaData.Create;
  with MetaData do
  begin
    with TableDefs.AddTableDef('Person','P',[tpUpdateable,tpInsertable]) do
    begin
      with AttributeDefs do
      begin
        aDef := AddDef('Person_ID',ftInteger,'ID',ftInteger,
                       [apPrimaryKey,apServerGeneratedBeforeInsert]);
        OID := ThcOID.Create('Person',[aDef]);
        AddDef('FirstName',ftString,'FirstName',ftString,[apRequired],30);
        AddDef('LastName',ftString,'LastName',ftString,[apRequired],30);
      end; //with
    end;  //with
  end;  // with
  ObjectRegistry.RegisterObject(TPerson,MetaData);
end;

The lifetime of the MetaData is managed for you, but since the Register procedure is not a callback, you must create the MetaData object.  Metadata contains the definition of the tables where the object attributes are stored and their relationships.  So you begin by adding a Table Definition to the Metadata, specifying the RDBMS table and whether the framework is allowed to insert or update rows in that table.  Then you add Attribute Definitions which essentially specify the column and it’s type in the RDBMS and the corresponding attribute name and type in the OPF.  The OPF is capable of translating database types into internal Delphi types and OPF attributes to some degree.  More on that later…

The final line in every Register() routine should be a call to register the object with the framework.  Namely:

ObjectRegistry.RegisterObject(TPerson,MetaData);

Then somewhere in the project you need to call the Register class method for all objects.  I typically do this in the initialization section of the same unit in which the business object is defined.  In the finalization section you should also call the UnRegister method, although technically this is not necessary since the application termination will reclaim the memory.

initialization
  TPerson.Register;

finalization
  TPerson.UnRegister;

In hcOPF a ThcAbstractFactory descendant is the object that is responsible for persisting a ThcObject descendant. To get a factory instance you request it from a FactoryPool.  All applications must contain 3 core objects:  a FactoryPool, a SQLMapper, and a TransactMgr.  Normally you would drop these components onto your main datamodule and hook them up like this:

The FactoryPool knows about the different factory classes available for persistence.  In this case, IBX has been chosen as the DAL for a FireBird database.  Once the datamodule has been created and the database is connected, you can load and persist objects as you wish.

For a simple example on using the framework with standard Delphi controls see the following.  This application has a rather contrived database structure used to show lazy loading (even though it’s triggered immediately) with a child object.  This example also shows how to display content from two different objects on the same form, and persist them in a single transaction.  The database included was created with Firebird 2.0.  I have included a script as well so you can use Interbase if you prefer.  I also included the EXE in case you want to run the application without setting up the framework to compile the project.