TPersistent.com

One of the newest ORM entries into the commercial market is EntityDAC from Devart.  If you are a Delphi developer you have probably heard of Devart, even if you haven’t actually used one of their products.  The company has been developing Delphi based technology since 1997 when they released their Oracle Data Access Components (ODAC).  Devart has specialized in data access related technologies, and since Delphi has predominantly been used to develop database centric applications, if you haven’t heard of Devart, chances are you were living under a rock somewhere.

While there are numerous ORM/OPF solutions available for Delphi, only a few use the latest language features of Object Pascal.  hcOPF, TiOPF, InstantObjects, DObject were all conceived prior to the appearance of generics, anonymous methods, and the new RTTI which has opened the door to dependency injection, and other modern best practices previously not available to Delphi developers.  That is not to say that none of these frameworks have adopted newer language features, just that they were not initially written with them or require newer versions of Delphi.  mORMot is an exception.  Of the current ORMs (if I missed one please let me know), only TMS Aurelius, DORM and now EntityDAC require a later version of Delphi (2007 and up).

In August 2014, Devart released the first version of EntityDAC for the Delphi platform.  EntityDAC builds on 18 years of expertise Devart has acquired developing database drivers and other ORM related products such as LinqConnect, EntityDeveloper (a designer for DB models) and dotConnect.  There have been several updates since it’s release, so unlike many open source solutions, you know it is actively being improved.

EntityDAC uses an object TDataSet descendant to present data, and recommends the usage of data aware controls to enforce validation so for Delphi developers used to using TDataSets it is as close to a drop in replacement as you can get.  EntityDAC supports Lazy Loading, Code-First, Model-First, or DataBase-First design, and is well documented.

One of the coolest features that I always wanted to implement for hcOPF was visual object design.  A complementary product; Entity Developer is bundled with EntityDAC.  It allows you to reverse engineer a database and create Delphi business objects as well as designing them from scratch.  It can then generate your model code so you can immediately start consuming your business objects.  Entity Developer is a Visual Studio shell based application that is also capable of using T4 templating supported in that IDE, so you can tweak the code generation as you see fit.

One of the questions I debated with other developers when I first wrote hcOPF was whether the ORM should support PODO (Plain Old Delphi Objects aka TObject descendants) as well as those descending from my base class ThcObject.  Limited RTTI precluded that possibility back in Delphi 3, and I have even interfaced ThcObject to experiment with enabling developers to use RefCounted PODO objects.  With EntityDAC you can use either TObject or TEntityObject as your base class.  It also supports using Delphi attributes to determine the database mapping or XML files.

Not only is the solution very flexible in terms of the workflows supported, the getting started Wizard makes it easy to get an application up and running,  and an example app showcases some of it’s capabilities.  The icing on the cake has to be LINQ which enables you to remove the SQL from your code and the coupling to the database it represents.

While I would love to dig deeper into EntityDAC, this is already getting to be a long post so perhaps I will write subsequent ones if there is enough interest.  Suffice to say, if you are looking for a commercial ORM solution backed by a company with almost 20 years experience in delivering high performance database centric solutions, I would recommend evaluating EntityDAC.

Get Awesomeness is a curated list of awesome Delphi frameworks, libraries, resources, and shiny things. Inspired by awesome-… stuff. It’s nice to have one stop shopping to open source frameworks of interest.  Certainly beats searching on source forge, Google code, GitHub and all the other sites.

Note that only open-source projects are considered. Dead projects are mainly ignored except for those which do not have alive analogs. Feel free to suggest other missing nice projects either by comments or pull requests.

Last night I I was reading some of the latest articles on Jon lennart Aasenden’s blog about Quartex. For those of you not following Jon on Google+ he is the founder of the DelphiArmy and the author of the Smart Mobile Studio IDE.  Quartex is an IDE for multiple languages (including Object Pascal) that Jon is working on that is to include the transcoding of languages.  IOW, you could take Delphi code and transcode it to C++ or C#.  The approach is to use the LDEF intermediate format, and from there you could either convert to a different language or compile it into binary code.

Quartex will be built on Jon’s cross platform framework and experience, and he is looking for others to join in. If the effort proves successful there could be another IDE choice other than Lazarus for XPlatform work. Imagine, a native IDE on the Mac compiling for OS/X or iOS, or a Linux IDE targeting Linus and Android. No more networking between machines with intermediate apps like PAServer, or using VMs to target mobile devices.

No more .NET subsystems like ErrorInsight that produces lots of false positives and never gets fixed. No more modelling support that no one uses, Refactoring that works some of the time and blows up out of memory or never comes back and no more half baked features that are abandoned. If Quartex ends up open source, and you find a bug that bugs you enough, you can fix it yourself or hire someone to do so.

Product direction would be a community decision, and both voting with your voice and your wallet would yield results. Object Pascal as a language would have a much better chance of surviving! It’s either that, or we continue with the high # of Quality Portal issue reports, and articles like this one that show how EMBT’s strategy is working in terms of quality (72 reports over the last 30 days without a single resolution).

I also thought I would mention that Marco has been on the EMBT quality portal. Good to see someone from EMBT providing an explanation of the complexity of the issue and what they’re doing.

What I always fail to understand is why developers (especially Delphi devs), tend to write programs using datasets with the normal RAD approach when all the benefits of Object Oriented Programming (OOP) are lost by doing so.  Datasets were used back when I wrote Clipper programs in 1992.  Clipper was not object oriented, so it’s like stepping back in time, and ignoring all the benefits that OOP has demonstrated over procedural programming.

When I was first learning OOP, I was taught “there are three pieces of PIE” where the acronym PIE stood for Polymorhpism, Inheritance, and Encapsulation.  You could of course say there are slightly more than 3 pieces (3.14) in a Π, or perhaps its just a case where the whole is greater than the sum of the parts.

IMO the characteristic or benefit most often overlooked when writing Delphi code is Encapsulation.  Encapsulation is what enables the de-coupling of classes, which affects your ability to maintain code over time with minimal breakage.

A good test of any code base is how easy it is to create a new application and re-use a class.  Try it, and you will soon find out what the prerequisites are, and how intertwined the code is.  Not only does this affect your ability to develop in an agile fashion, it means re-factoring the code for better performance is more difficult as well.  One of the most common issues with legacy Delphi code is that all the work is done in a single thread (the main VCL thread).  Progress reporting usually involves calls to Application.ProcessMessages() to ensure processing results are displayed in a timely fashion.  What this means is that the processsing takes longer, and is even more difficult to move it to a background thread because it is tied to the VCL message loop as well as other objects in the main thread.

If I had to write an application from scratch I would:

1) use an ORM whenever possible.  DataSets are compact and fast to retrieve.  Where they fall down is the encapsulation of data and behaviour (business logic) which changes over time.  ORM objects are more flexible, and they usually provide a validation framework as well as persistence.  Sooner or later datasets will force you to break the DRY principle in anything but a trivial application.

2) use MVVM.  While MVVM frameworks for Delphi are in their infancy, the concept can be implemented on a case by case basis. The idea is to keep as much code out of the View (form) as possible.  What you really need for MVVM is some kind of object/data binding.

3) implement your data processing on a secondary thread.  If you do so right off the bat, there is less of a chance that the data access will ever be tightly coupled with anything on the main thread, and parallelization becomes trivial.

4) use TDD to write at least the core classes in your onion architecture.  The foundation on which you build an application needs to be bullet proof, otherwise you’re just building a house of cards, and when problems arise, or changes are required the termites start to come out of the woodwork.  This also has the benefit that you can test and optimize the performance of each piece so once it’s all put together it should be as lean as possible.  TDD forces developers to document the code they write in the form of tests.

Delphi’s compiler tool chain may automatically eliminate some dead code, but with an older or “mature” product there are bound to be lots of units that are no longer used but are still present as part of a project or referenced indirectly by some other unit. Having a good gatekeeper on your team can help, as can these tools, but it can never replace a team that is concerned about keeping it lean.

For instance, I stumbled across code that was written to compensate for a database column not being a consistent length. That change management issue was addressed a long time ago, yet the code remained, running a very expensive query on Oracle for absolutely no benefit. Some people may not think that is a big deal, but multiply that by 1000 other such things in the main thread of an application and it can start to make Delphi programs look pretty sluggish.

If your team keeps units around because it ‘might’ need them, or older methods because not every call has been replaced, you might want to rethink that.  It’s easy to spend time maintaining code that may no longer be relevant.  Time means money, and there is no ROI if you are a code hoarder.  A VCS makes it easy to restore code in the event you ever need it, or disposed of it by accident.

In the 18+ years Delphi has been out, some rather large applications having been built.  Between the number of units developers are using, the size of the component sets installed in the IDE and the transition to the Galileo IDE with .NET subsystems, Delphi now requires more memory than ever.  What is interesting is that Delphi is still a 32 bit IDE even after release of the 64 bit compiler.  In the old days when Delphi was built using the Object Pascal compiler, one would have thought a 64 bit version would be forthcoming shortly after the compiler was available.

I would venture that most developers run a 64 bit version of Windows natively or in a VM for development so it begs the question as to why a 64 bit version of Delphi is not at least available.  Apple’s testing showed compiling their desktop software for 64 bit provided for about a 20% speed gain.  Pretty impressive for just a re-compile.

In addition to a performance benefit, 64 bit apps of course have access to more memory.  Considering the Out of Memory issues perhaps the additional memory would have at least delayed failure of the IDE.  Even more interesting is the fact that the current bds.exe is not Large Address Aware.

Delphi is not Large Address Aware

This could have been enabled by simply adding the {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} to the project.

If you are experiencing Out of Memory issues you could of course use a utility like PE Viewer to make the EXE large address aware, but then you would get directed here the next time you launch Delphi because the copy protection code would prevent it from launching.

Isn’t it about time that EMBT supported 64 bit OS/X apps, and a 64 bit version of Delphi or at least made Delphi Large Address Aware?

Recently I decided to try to incorporate Windows Job Objects into a project to artificially limit the amount of memory available to an application so I could test it’s behaviour under such circumstances.  Originally I had intended on using M$’s Application Verifier for that purpose, but it lacks any real documentation, and apparently does not provide this exact functionality (the low resource utilization provides random failures).

I thought I would use the JWSCL library since it has a class that wraps Job Objects.  Getting it to compile was unfortunately not so straightforward.

To start with, the JWSCL requires the JWA library, so you need to get that compiled first.  I had no luck compiling the downloaded version so I pulled the current source from the trunk.  Then I followed the READMEFIRST.txt file for the JWA library, but got a compilation error:

 E1025 Unsupported language feature: 'import of DLL symbol by ordinal' "

Googling this I stumbled across a reply to a post from Peter Below explaining “This is caused by a change in defaults for the project options. Call up the options dialog, select the Output - C/C++ page, and make sure the “C/C++ output file generation” entry is set to “Generate DCUs only”. I toggled the option, and voila! The JWA compiled.

Then I tried the JWSCL and got a “[dcc32 Error] JwsclLsa.pas(121): E2010 Incompatible types: ‘Cardinal’ and ‘NativeUInt’. Seems that a THandle is used for some definitions and equates to a NativeInt in XE4, but in other places handles are defined as Cardinals. TJwSecurityLsa uses a mixed definition of a property and changing it to use THandle just opens up the proverbial rabbit hole.

I ended up abandoning my attempt to get JWSCL compiling in part because I downloaded another implementation of the Job Object API from here.  Thanks ZigiZ.  It’s unfortunate that such libraries are so difficult to maintain for multiple compilers. I know from personal experience with hcOPF that it is very difficult to change a library and maintain a good user experience for developers using a multitude of Delphi versions.

Now if only I could find out if it is possible to add a process to a job from that process (I get a security error)

I have seen code where an object pointer is stored in each row of a TDataSet, and referenced by code.  So skipping the question of the validity of such code, how do you support X86 and X64 bit code, especially if you use persistent TFields (use Delphi’s Form Designer RAD approach for the UI)?

For 32 bit development a TIntegerField works just fine.  Compile for Win64 (the OS/X compiler still doesn’t support 64 bit code) and everything seems fine until you run the app and exercise the code to find an AV.

A NativeInt scalar type is variable depending on the word size of the CPU, but there is no equivalent TField descendant (TNativeIntField), so you are forced to choose a different TNumericField descendant depending on your target (32 or 64 bit) or one that can support both.  The TLargeIntField uses a largeint (equivalent to an Int64) for internal field storage so it is the only possible choice that could support both targets and still provide design-time support.  The overhead of such an approach is questionable.  Otherwise, if you create the fields in code I would suggest conditional compilation using the CPUX64/CPUX86 directives.

If you need to support 64 bit targets I would also suggest development using Win64 so you don’t have to test each pathway for a 32 bit application to ensure it runs fine as a 64 bit EXE.

Compiler Hints and Warnings are an important indicator of the quality of the code you are compiling.  It dismays me that I have often seen a ton of compiler warnings on commercial component suites when I compile them.  It makes you question the care taken when writing the code.

Compiler Hints and Warnings should be treated as potential coding errors.  Each one should be addressed to ensure it will not cause a errant behaviour at run-time, and the code in question is properly structured.  It is good to see I am not alone in this view.  So take a hint (pun intended), and clean up your hints and warnings.

Sometimes this isn’t so easy.  For instance, if  component requires a given unit, for example Vcl.ImgList, as the TcxCustomTreeList does, it will add the unit to the uses clause automatically.  You might then get a  warning like  W1000 Symbol ‘TImageIndex’ is deprecated: ‘Use System.UITypes.TImageIndex’. Unfortunately, adding System.UITypes and removing Vcl.ImgList just results in the IDE re-injecting Vcl.ImgList back into the uses clause and reporting the same warning.  I really don’t want to turn off this warning globally so the best option that I know of is to add {$WARN SYMBOL_DEPRECATED OFF} to the interface section of the unit in question.  While this solution may not be ideal, it allowed me to “address” the warning so new compiler hints and warnings that may actually indicate a flaw in the code do not get lost in a multitude of messages.  Clean code is less likely to be flawed code.

It’s been my experience that just like in sports where you have a goal tender, every development team needs a gate keeper or goal tender for their VCS to ensure nothing gets in that they don’t want.  The role of the gatekeeper in the team is not only to manage the promotion/demotion of DEV code to QA on through to PROD (production), but also to ensure the code meets syntax standards, and more importantly architectural standards or conventions.

Even if the standard way of doing things is less than optimal, consistency is king.  If a slower evolutionary method of evolving the code is not possible, at least with consistent code, you can transform it en masse at the beginning of a release cycle and dedicate a lot of QA time to ensure you didn’t break anything.  Developers who understand the “normal” way of doing things can maintain and write code comfortably, with confidence that it will work.

I have seen consultants brought in to author a new system and use it as an opportunity to play with new architecture and methodologies.  While they usually deliver the results, the code most often cannot be effectively supported by on-site developers because it is so radically different from what they know.  The same thing can happen on a team with developers at different skill levels, and different views on the latest design patterns and methodologies.

The important thing is that ‘there is no “I” in TEAM’.  Everyone needs to be able to understand and support all the code.  If an architectural pattern makes software harder to understand, then the benefits of using that pattern may be outweighed by the maintainability of the code by the team.  The team needs to decide as a whole on coding standards and practices.

One example of this is the heavy use of interfaces.  Interfaces are great for decoupling code, but at the same time it makes it harder to follow since the implementer of the interface is “disconnected”.  Using the Delphi IDE to navigate through code, by Ctrl+clicking in order to understand it, doesn’t work as it does with classes.  This isn’t a shortfall of the IDE.  There is simply no way it can know what objects are used to implement that interface (especially if you use injection).

The quality and maintainability of your code is directly related to the strength of the gatekeeper and the cohesion of the team.  Better teams build better code…