TPersistent.com

CJH made some valid points about my last article.

I guess I really wasn’t that clear on how my story showed the benefits of Open Source.  Let me attempt to rectify that.

Open Source can mean different things.  You can read most of the VCL source, and that has numerous benefits that could not be achieved otherwise.  The VCL source is a kind of restricted open source since the EULA restricts you from doing all that much with it.  It’s still very useful in that you can find out what method to override to achieve the desired functionality, or find a more efficient algorithm.  You’re not reliant on documentation that can be grossly incomplete or incorrect.  Nothing is better at teaching you how to code, than reading good code, and if it makes it into a commercial release it should be good right?  Well perhaps not as good as it could be, because how else would you ever be able to get over an order of magnitude of a performance gain.  So point #1 is that without the ability to read the RTL/VCL source an add-in like DelphiSpeedUp would have been more difficult to craft.

I have collected many Delphi books over the years, and some of them like the “Hidden Paths of Delphi 3″ by Ray Lischner really opened up the Open Tools API.  Information collected by experimentation, talking to Borland engineers, and no doubt reverse engineering that was shared within the community helped make the IDE a platform for innovation.  Out of that information came products like CodeRush and Castalia.  Open Source projects like GExperts, CN-Pack produced add-ins that provide functionality that should be part of the commercial offering.  So point #2 is that EMB should recognize the benefits that Open Source has brought to developers who use their product, and facilitate these projects/products so they can thrive, and integrate better with the IDE or toolset.  CodeGear started down this path by making FastMM4 the default memory manager, but there is so much more than could be done.  This might involve shipping with, and being part of the native Delphi install.  This might mean having their own open source hosting service for Delphi projects.  If it makes sense for RemObject or Gurock software why not EMB?  What about dedicated people to work on open source projects and incorporate the results into the VCL?

Point #3 is that  having the VCL source code likely prompted some of the Delphi developers to read it and say “we can do better”, leading to projects like FastMM4, FastCode and KOL.  Delphi Speedup leverages the FastCode project to do it’s magic.

It would be nice if EMB would open up more areas, and accept contributions directly from the community, but I don’t see how that is possible unless they entertain a free version.  Not too many people run Darwin on their Mac, and no one will contribute code for free and allow someone else to benefit financially from it without giving back.

That was the whole point I was trying to make (albeit indirectly).  I think EMB needs to open source the compiler leveraging LLVM (look at Apple’s XCode 4), standardize and publish the language specification for all to use, and open up the IDE so tools like Castalia, and GExperts don’t have to implement their own code parser.

Borland realized some of the benefits of Open Source by releasing most of the VCL source code.  It didn’t hurt their business. Even Firebird helped increase the number of developers using the technology, and when they wanted features like SMP support and wire encryption, they turned to Interbase.  Developers are a unique audience when it comes to Open Source.  We can actually contribute in a meaningful fashion, and EMB doesn’t have the resources or time they need to hit the necessary targets.  If EMB wants to see Delphi everywhere they are going to have to compete with Apple’s XCode 4, and provide a 64 bit compiler ASAP.  I think they can use all the help they can get, and that makes Open Source more appealing than ever.

I have always enjoyed the speed of Delphi’s compiler.  It definitely sets Delphi apart from other native toolsets like Visual C++.  Over the last several weeks work on my latest project was grinding to a halt because for some reason the Delphi 7 compiler was taking any where from 3 to 3m22s to compile my project.

I’m sure many people out there would simply say “upgrade” to solve the problem.  That’s where it gets a little complicated.  I had evaluated Delphi 2010, and Delphi 2007 early on during project development.  The existing product version is in fact written with Delphi 2007, but I chose to use Delphi 7 instead.  Why?  In one word, CodeRush.  CodeRush is a very old product, but provides templating and refactoring support that is still beyond what is offered in the native IDE.  I’m also very comfortable with it.  The problem with “upgrading” is of course all the components used.  I have source for them, but migrating all of that, including design-time code can be rather onerous.

Besides, as a programmer, you need to find out why.  My other projects including hcOPF compile lightning fast as always.  So I contemplated what made this specific project so unusual.  Currently it only contains 64 units, uses the DevExpress grid suite (of course), the JCL/JVCL, hcOPF and a smattering of other small components.  One of the major differences was using the JCL/JVCL, and I know that it sucks in quite a large number of units the way it’s organized, but I still didn’t think that could be the cause.

I finally decided to post on the embarcadero newsgroups.  The Delphi community is still pretty vibrant, and there are lots of folks out there who have been using the product from day 1.  I got several suggestions including:

Put only .dcu, .dfm, .res, .inc file in the LIBRARY PATH, and .pas files only in the Browsing Path.

This did seem to help, but compilations were still about 3 minutes.

Then Marius suggested I look at Delphi SpeedUp.  I had ran across it earlier in my googling for performance improvements, but never considered it a solution to my problem because my performance problem was project specific.   Was I ever wrong!  Compilation times went from over 3 minutes to 11 seconds.

So this is the second time I must thank Andreas Hausladen; once for updating the JCL/JVCL to use the LIBSUFFIX directive to make it easier to manage my hcOPF packages for different Delphi versions, and now for Delphi SpeedUp.  Thanks also to the contributors to the FastCode project.

This just goes to show what Open Source can do for commercial products.  I think Embarcadero could leverage Open Source more effectively to build out the VCL, and improve the IDE.  Even the compiler could benefit from Open Source projects like LLVM.  Apple seems to have prospered by leveraging Open Source…

As I’ve already mentioned, I don’t think any good OPF should force the developer to choose to implement their entire UI using objects.  Sometimes it’s easier and requires less CPU time to simply use a dataset.

Today I found myself in just that situation.  I needed to add amounts owed on account for a particular reason in a dialog.  The dialog also displayed all previous amounts owed for the same reason in a read only grid.  I didn’t want to load a list of amounts owed, and then have to filter it to those owed for the reason in question, or load a separate list at some point for all amounts owed regardless of the reason.  This would have consumed more memory and CPU resources than necessary, and the benefit of the OPF is the ability to bind to the UI controls, validate the object, and persist it with minimal effort in a database independent fashion.  An added benefit of using an OPF is that because all updates & inserts are done through the objects, you don’t need to refresh them from the database unless they could become stale.  If you’re not going to update the data, there is little point in using an object list.

The challenge is that I still want to be database independant so I use the IhcQuery interface rather than creating an instance of a query for the DAL I am employing directly.  So I needed a way to attach a dataset I loaded using a SQL statement through the IhcQuery interface to a DataSource which only accepts a TDataset class.  Since Delphi 7 doesn’t provide the ability to cast an interface to the object that implements it, I did a quick google to find this.  The Delphi Magazine was a great read.  Thankfully I bought the CDs and there is still a lot of the material on the Net.  Thanks to Hallvard VassBotn, this code is now part of hcInterfaceUtils.

I tried to get together with Nick Hodges to discuss some editor enhancements I would like to see in Delphi while I was in San Fran for WWDC2010.  Most of these features are based on my experience with CodeRush (yes I’m an addict).  Unfortunately, the timing didn’t work out this time, but it got me started looking for suggestions, and once you’re in the mindset, they’re endless.

For instance, have you ever added new units without adding them to a project file and got the message:

[DCC Warning] HengenOPFCore.dpk(101): W1033 Unit ‘hcFloatAttribute’ implicitly imported into package ‘HengenOPFCore’

Wouldn’t it be nice to be able to Rt. Click on the message and choose ‘Add to Project’?

I am making a list in case Nick and I can hookup sometime in the future.  What Editor enhancements would you like to see in the base product?

I ran across this post today and was outraged that it’s exactly what I posted here two days earlier.  I like that it’s on another Delphi blog, but why copy my content?  Looks like the “author” may be pilfering other people’s posts on Delphi subjects as well….so beware.

Today I was working on a legacy product that required new functionality and came across a use case where an OPF was a far easier solution than using TDataSet descendants.  The current product was implemented using ADO datasets and the new one, using hcOPF.  I needed to add the same functionality to both versions with minimal changes.

I decided to use datasets because the current product needed to be deployed before the one in development, and it used datasets.  The form I needed to implement contained a grid where users could toggle a single boolean field - a simple UI.  I thought there was no reason to add the overhead of hcOPF into the mix.  I was wrong!

The first wrinkle was that the data came from a SQL Server view and the column that was to be updated  belonged to a table added to the view by an outer join.  So an actual record in the source table may not exist.  While the view was updateable in SQL Server, ADO considered the TField ReadOnly.  I thought I would add a calculated field to the dataset, update the calculated field in the cxGrid, and use that value to call a StoredProc to update the underlying database column (it handles the problem whether the record exists or not).  Unfortunately, calculated fields are considered ReadOnly as well.  Using the OnCalcFields event was also problematic because I had to initialize the calculated field to the value of the column in the view to start with.

The solution was to use define a new ThcObjectList class with a custom Load method, and use a query to load the object list.  I also implemented a custom Write method in the ThcObject descendant used in the objectlist that checked if the original source column was Null.  If the source column was Null I knew I needed to insert a record into the source table, and if not, update the table with the value of the boolean attribute.

I’ve heard developers say they don’t like ORMs or OPFs because you have to learn how they work in order to leverage them effectively and there is always some scenario where the OPF boxes you in.  I would agree that there is a learning curve as with any component suite or library, but I don’t agree that an OPF limits you in any way.  You should be able to use a mix of an OPF with datasets, choosing whatever approach makes more sense.  It’s been my experience that an OPF provides a lot of necessary and valuable functionality that can shorten up front development time, and most importantly reduce maintenance over the lifetime of the software.

Maybe they were just using the wrong OPF…