Making Delphi 64 Bit/Large Address Aware

August 20th, 2014

In the 18+ years Delphi has been out, some rather large applications having been built.  Between the number of units developers are using, the size of the component sets installed in the IDE and the transition to the Galileo IDE with .NET subsystems, Delphi now requires more memory than ever.  What is interesting is that Delphi is still a 32 bit IDE even after release of the 64 bit compiler.  In the old days when Delphi was built using the Object Pascal compiler, one would have thought a 64 bit version would be forthcoming shortly after the compiler was available.

I would venture that most developers run a 64 bit version of Windows natively or in a VM for development so it begs the question as to why a 64 bit version of Delphi is not at least available.  Apple’s testing showed compiling their desktop software for 64 bit provided for about a 20% speed gain.  Pretty impressive for just a re-compile.

In addition to a performance benefit, 64 bit apps of course have access to more memory.  Considering the Out of Memory issues perhaps the additional memory would have at least delayed failure of the IDE.  Even more interesting is the fact that the current bds.exe is not Large Address Aware.

Delphi is not Large Address Aware

Delphi is not Large Address Aware

This could have been enabled by simply adding the {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} to the project.

If you are experiencing Out of Memory issues you could of course use a utility like PE Viewer to make the EXE large address aware, but then you would get directed here the next time you launch Delphi because the copy protection code would prevent it from launching.

Isn’t it about time that EMBT supported 64 bit OS/X apps, and a 64 bit version of Delphi or at least made Delphi Large Address Aware?

Compiling the JWA and JWSCL

August 14th, 2014

Recently I decided to try to incorporate Windows Job Objects into a project to artificially limit the amount of memory available to an application so I could test it’s behaviour under such circumstances.  Originally I had intended on using M$’s Application Verifier for that purpose, but it lacks any real documentation, and apparently does not provide this exact functionality (the low resource utilization provides random failures).

I thought I would use the JWSCL library since it has a class that wraps Job Objects.  Getting it to compile was unfortunately not so straightforward.

To start with, the JWSCL requires the JWA library, so you need to get that compiled first.  I had no luck compiling the downloaded version so I pulled the current source from the trunk.  Then I followed the READMEFIRST.txt file for the JWA library, but got a compilation error:

 E1025 Unsupported language feature: 'import of DLL symbol by ordinal' "

Googling this I stumbled across a reply to a post from Peter Below explaining “This is caused by a change in defaults for the project options. Call up the options dialog, select the Output - C/C++ page, and make sure the “C/C++ output file generation” entry is set to “Generate DCUs only”. I toggled the option, and voila! The JWA compiled.

Then I tried the JWSCL and got a “[dcc32 Error] JwsclLsa.pas(121): E2010 Incompatible types: ‘Cardinal’ and ‘NativeUInt’. Seems that a THandle is used for some definitions and equates to a NativeInt in XE4, but in other places handles are defined as Cardinals. TJwSecurityLsa uses a mixed definition of a property and changing it to use THandle just opens up the proverbial rabbit hole.

I ended up abandoning my attempt to get JWSCL compiling in part because I downloaded another implementation of the Job Object API from here.  Thanks ZigiZ.  It’s unfortunate that such libraries are so difficult to maintain for multiple compilers. I know from personal experience with hcOPF that it is very difficult to change a library and maintain a good user experience for developers using a multitude of Delphi versions.

Now if only I could find out if it is possible to add a process to a job from that process (I get a security error) :-(

Using Object Pointers in a DataSet TField

July 25th, 2014

I have seen code where an object pointer is stored in each row of a TDataSet, and referenced by code.  So skipping the question of the validity of such code, how do you support X86 and X64 bit code, especially if you use persistent TFields (use Delphi’s Form Designer RAD approach for the UI)?

For 32 bit development a TIntegerField works just fine.  Compile for Win64 (the OS/X compiler still doesn’t support 64 bit code) and everything seems fine until you run the app and exercise the code to find an AV.

A NativeInt scalar type is variable depending on the word size of the CPU, but there is no equivalent TField descendant (TNativeIntField), so you are forced to choose a different TNumericField descendant depending on your target (32 or 64 bit) or one that can support both.  The TLargeIntField uses a largeint (equivalent to an Int64) for internal field storage so it is the only possible choice that could support both targets and still provide design-time support.  The overhead of such an approach is questionable.  Otherwise, if you create the fields in code I would suggest conditional compilation using the CPUX64/CPUX86 directives.

If you need to support 64 bit targets I would also suggest development using Win64 so you don’t have to test each pathway for a 32 bit application to ensure it runs fine as a 64 bit EXE.

A Hint About Warnings

July 22nd, 2014

Compiler Hints and Warnings are an important indicator of the quality of the code you are compiling.  It dismays me that I have often seen a ton of compiler warnings on commercial component suites when I compile them.  It makes you question the care taken when writing the code.

Compiler Hints and Warnings should be treated as potential coding errors.  Each one should be addressed to ensure it will not cause a errant behaviour at run-time, and the code in question is properly structured.  It is good to see I am not alone in this view.  So take a hint (pun intended), and clean up your hints and warnings.

Sometimes this isn’t so easy.  For instance, if  component requires a given unit, for example Vcl.ImgList, as the TcxCustomTreeList does, it will add the unit to the uses clause automatically.  You might then get a  warning like  W1000 Symbol ‘TImageIndex’ is deprecated: ‘Use System.UITypes.TImageIndex’. Unfortunately, adding System.UITypes and removing Vcl.ImgList just results in the IDE re-injecting Vcl.ImgList back into the uses clause and reporting the same warning.  I really don’t want to turn off this warning globally so the best option that I know of is to add {$WARN SYMBOL_DEPRECATED OFF} to the interface section of the unit in question.  While this solution may not be ideal, it allowed me to “address” the warning so new compiler hints and warnings that may actually indicate a flaw in the code do not get lost in a multitude of messages.  Clean code is less likely to be flawed code.

Every Team needs a Goaltender

July 22nd, 2014

It’s been my experience that just like in sports where you have a goal tender, every development team needs a gate keeper or goal tender for their VCS to ensure nothing gets in that they don’t want.  The role of the gatekeeper in the team is not only to manage the promotion/demotion of DEV code to QA on through to PROD (production), but also to ensure the code meets syntax standards, and more importantly architectural standards or conventions.

Even if the standard way of doing things is less than optimal, consistency is king.  If a slower evolutionary method of evolving the code is not possible, at least with consistent code, you can transform it en masse at the beginning of a release cycle and dedicate a lot of QA time to ensure you didn’t break anything.  Developers who understand the “normal” way of doing things can maintain and write code comfortably, with confidence that it will work.

I have seen consultants brought in to author a new system and use it as an opportunity to play with new architecture and methodologies.  While they usually deliver the results, the code most often cannot be effectively supported by on-site developers because it is so radically different from what they know.  The same thing can happen on a team with developers at different skill levels, and different views on the latest design patterns and methodologies.

The important thing is that ‘there is no “I” in TEAM’.  Everyone needs to be able to understand and support all the code.  If an architectural pattern makes software harder to understand, then the benefits of using that pattern may be outweighed by the maintainability of the code by the team.  The team needs to decide as a whole on coding standards and practices.

One example of this is the heavy use of interfaces.  Interfaces are great for decoupling code, but at the same time it makes it harder to follow since the implementer of the interface is “disconnected”.  Using the Delphi IDE to navigate through code, by Ctrl+clicking in order to understand it, doesn’t work as it does with classes.  This isn’t a shortfall of the IDE.  There is simply no way it can know what objects are used to implement that interface (especially if you use injection).

The quality and maintainability of your code is directly related to the strength of the gatekeeper and the cohesion of the team.  Better teams build better code…

EMBT gets a Swift Kick in the Assetts

June 4th, 2014

In case you haven’t heard yet, Apple has created a new language meant to be more friendly than Objective C.  They call it Swift, and it is aptly named judging from the benchmarks which show it to be significantly faster than Objective C even though it uses the same run-time and is compiled using the LLVM toolchain.

What does this mean for Delphi?  Perhaps a few developers looking to produce truly native (UI and code) will take another look at Apple’s development tools that they already have to access to as part of the development program.

ReEnabling the GExperts Procedure List (Ctrl +G)

May 21st, 2014

If you are like me, you use GExperts in your IDE.  IMHO much of the functionality GExperts offers should be part of the stock IDE.  Once you start using it, there is no going back.  Many thanks to Erik Berry and company for putting all this together and keeping it supported through so many releases of Delphi.

Sporadically I have noticed that Ctrl+G in XE4 with GExperts installed stops working.  If I restart the IDE it would again work for a while, but eventually I was frustrated enough to investigate the issue further.  Using GExpert’s IDE Menu ShortCuts option I navigated through the menu looking for menu items using the same shortcut.  I stumbled across the Find Original Symbol menu item which uses the same shortcut (it would be nice to be able to search for a shortcut but I did not see such an option).  I re-mapped it to a custom shortcut since I am used to using Ctrl+G for the GExperts Procedure List.  This worked temporarily, but then I noticed Ctrl+G was still shown in the Find Original Symbol menu option.  I re-started the IDE and it then showed the new shortcut, and Ctrl+G launches the procedure list as expected.

I hope this helps in case you are experiencing the same issues.

MSBuild MSB6003 Work Around

April 16th, 2014

One of the QC items not fixed in the XE6 release is 98008, probably because it is still in the Reported state even though it was submitted August 23, 2011. I have previously blogged about the state of QC, so I won’t get into that issue again, suffice to say I hope the QPS initiative continues, and QC incidents get reviewed.

I have been encountering the same error as QC98008 because between DevX, TeeChart, and TMS the search paths for units (stored in OPTSETS) has exceeded the 32,000 character limit primarily because the same paths are used multiple times by MSBuild when constructing the build command.  It was included in the ResourcePath, IncludePath, UnitSearchPath and ObjectPath.  Since no OBJ files were in use, and resources are included in the project files, we simply changed the CodeGear.Delphi.Targets file located in the Delphi Bin folder, setting both the ResourcePath and ObjPath to an empty string (lines 192 & 194) as follows:

ResourcePath=""
IncludePath="$(IncludePath)"
ObjPath=""

This resolved the duplicate path issue that resulted in the exceedance of the 32000 character limit.

XE6 Released with some Interesting Features

April 16th, 2014

I attended the webinar today announcing Delphi XE6. It’s been a long road for the EMBT dev team to provide both C++ and Object Pascal support for iOS, Android, OS X, and Windows in the same environment. Kudos to them for accomplishing that feat, and incorporating over 2000 bug fixes into the XE6 release.  The article lists 500+ fixes for customer reported issues.  I assume the remaining 1500+ items were issues found internally.  Using QC for Windows I found that there are already 17 bug reports for XE6 build 20.0.15596.9843.  Unfortunately I cannot get a listing of all the issues fixed because querying using 20.0 or the full build # in Resolved in Build field yields 0 results.

I am looking forward to installing a trial in a VM, and checking out the latest “Key New Features” - Quality, performance and stability!  It was great to see that the VCL got some love and of special interest to me was the tethering capabilities for mobile devices.  I can’t wait until 64 bit OS X apps and Linux are supported, but the product is definitely heading in the right direction.

Using CodeSite to Spot Peformance Optimization Opportunities

April 14th, 2014

I have been using CodeSite for a long time now (since v2 days), but I am still continually impressed by the product and it’s capabilities.  If you want to do something with your log data, chances are CodeSite has you covered, you just need to figure out how it is implemented.

Today, I wanted to see if I could filter on the TimeStamp column shown as a TimeDifference in order to only see log entries where the difference between one line and the next was over a second.  I figured this would show me potential opportunities for code optimization.  Initially I thought I would have to write some sort of log mining tool since there didn’t seem to be any capability to filter the log in this manner.  Developers often immediately jump to “I can solve this by writing some code”, but instead I asked my trusted friend Google.

Google showed me some release notes from a few versions back that documented a new feature in 4.62.  I hadn’t used this version so I had no idea the following was possible:

5. The CodeSite Viewers now support Time Difference Navigation. Specifically, a
   new mode has been added to the Message Navigation buttons (First, Previous,
   Next, and Last) to allow navigating to a message where the amount of time
   that has passed since the previously logged message exceeds some threshold.
   This new navigation mode is extremely helpful is locating time gaps in a
   message log.

   To use Time Difference Navigation, simply click on the "Browse for ..."
   drop-down button located between the Previous and Next navigation buttons.
   At the end of the popup menu, a new Time Difference menu item is available.
   Clicking the Time Difference menu item results in a dialog box asking for a
   threshold value (in milliseconds). Clicking OK closes the dialog box and
   puts the Viewer into Time Difference Navigation mode.  You can then use the
   First, Previous, Next, and Last navigation buttons to locate messages where
   the Time Difference of that message exceeds the specified threshold.

   To toggle back into normal Message Navigation mode, simply select the Time
   Difference menu item again from the drop down list of message types. The
   previous Message Navigation settings will be restored.

This is a sweet suite feature….thanks Ray!