Encapsulate Your Data

What I always fail to understand is why developers (especially Delphi devs), tend to write programs using datasets with the normal RAD approach when all the benefits of Object Oriented Programming (OOP) are lost by doing so.  Datasets were used back when I wrote Clipper programs in 1992.  Clipper was not object oriented, so it’s like stepping back in time, and ignoring all the benefits that OOP has demonstrated over procedural programming.

When I was first learning OOP, I was taught “there are three pieces of PIE” where the acronym PIE stood for Polymorhpism, Inheritance, and Encapsulation.  You could of course say there are slightly more than 3 pieces (3.14) in a Π, or perhaps its just a case where the whole is greater than the sum of the parts.

IMO the characteristic or benefit most often overlooked when writing Delphi code is Encapsulation.  Encapsulation is what enables the de-coupling of classes, which affects your ability to maintain code over time with minimal breakage.

A good test of any code base is how easy it is to create a new application and re-use a class.  Try it, and you will soon find out what the prerequisites are, and how intertwined the code is.  Not only does this affect your ability to develop in an agile fashion, it means re-factoring the code for better performance is more difficult as well.  One of the most common issues with legacy Delphi code is that all the work is done in a single thread (the main VCL thread).  Progress reporting usually involves calls to Application.ProcessMessages() to ensure processing results are displayed in a timely fashion.  What this means is that the processsing takes longer, and is even more difficult to move it to a background thread because it is tied to the VCL message loop as well as other objects in the main thread.

If I had to write an application from scratch I would:

1) use an ORM whenever possible.  DataSets are compact and fast to retrieve.  Where they fall down is the encapsulation of data and behaviour (business logic) which changes over time.  ORM objects are more flexible, and they usually provide a validation framework as well as persistence.  Sooner or later datasets will force you to break the DRY principle in anything but a trivial application.

2) use MVVM.  While MVVM frameworks for Delphi are in their infancy, the concept can be implemented on a case by case basis. The idea is to keep as much code out of the View (form) as possible.  What you really need for MVVM is some kind of object/data binding.

3) implement your data processing on a secondary thread.  If you do so right off the bat, there is less of a chance that the data access will ever be tightly coupled with anything on the main thread, and parallelization becomes trivial.

4) use TDD to write at least the core classes in your onion architecture.  The foundation on which you build an application needs to be bullet proof, otherwise you’re just building a house of cards, and when problems arise, or changes are required the termites start to come out of the woodwork.  This also has the benefit that you can test and optimize the performance of each piece so once it’s all put together it should be as lean as possible.  TDD forces developers to document the code they write in the form of tests.

12 Responses to “Encapsulate Your Data”

  1. A. Bouchez Says:

    This is exactly one of the points what I try to emphasize during all training I give to Delphi developers: even if RAD is implemented by OOP, RAD lost all the OOP benefit from the application point of view. RAD reduces the code to be procedural oriented.
    RAD is great for the UI, and one great advantage of Delphi, but once you mix logic and UI (i.e. write logic in your RAD events), your code begins to be unmaintainable.
    This is especially true today, when we target multi clients: Desktop apps, mobile apps, web apps, web sites… with the same
    To be accurate, TDD is not the root concept of “onion architecture”. SOLID principles may be, especially the fact that all objects and interfaces should be uncoupled, so testable and injectable.
    In fact, the “onion architecture” was popularized by the Domain Driven Design modeling pattern.
    I agree with your points of how to “write an application from scratch”. And in this case, I would like to state that our Open Source mORMot framework has all the features you described… and more. http://mormot.net
    And if you have to maintain a legacy application, DDD may also help. In fact, you can use ORM, MVVM, TDD, DDD with implementing the new features in SOA services, consumed by your legacy app. Then you can introduce DDD only for your core business code.
    SOA is worth investigating for any kind of application (new or legacy), together with DDD.
    We tried to present a lot of those architectural/design concept in our documentation, and slides. See http://blog.synopse.info/post/2014/04/18/Introducing-mORMot-s-architecture-and-design-principles

  2. Stefan Glienke Says:

    Simple answer: because Delphi itself and most of the “experts” propagate that way of doing so. FireDAC, DataSnap, all dataset centric just saying. LiveBindings, how effing complicated it is to bind a simple PODO or a list of them to some controls.
    We’ve been doing it since ‘95 so we keep doing it that way.

    Same with TDD. Or rather unit testing at all. It’s treated like the redheaded step child. (I am guilty of that aswell as I still have to force myself into writing tests before hacking down some stuff -.-) Especially TDD requires some concept upfront but with RAD you can just start and create a bloody mess by dropping all your stuff on the form (or data module if you are a “pro”).

    As for MVVM that’s been a topic I have been on and off for a while now and want to get that into my focus again this year. So I would welcome sharing some opinions on that.

  3. Leonardo Herrera Says:

    Well, a dataset-powered grid is just a beautiful thing that ORMs have a hard time replicating. Live updating data in the same way one edits a spreadsheet is a really good user experience.

    I sell an application that uses a grid as its main view. But all data operations are done using objects and a half-assed DAO layer. But I still cannot replace my main gridview because have you seen DevExpress grids? They are just beautiful.

  4. D. Davis Says:

    The last large project I worked on in the mid naughties had an inhouse (micro) ORM. The frustration began when new hires (all senior) just couldn’t get past the dataset mentality, and no amount of explaining could seem to get the idea across. 10 years on, still no out of the box ORM /sigh/.

  5. Michael Says:

    A relational database is not a mechanism to store application objects or let’s say has not been designed that way in a first place. It hosts an information model.

    Binding that data to the application is another story. I agree that binding objects in an application does allow a more flexible handling. The applications data model is not the information model. It’s a view to the information stored in the pool.

    If you process data storage and retrieval in an extra thread you simply send messages to the database. Screen = Message. Old fashioned idea wrapped and mashed up in clothes of object orientation.

    If you go beyond that agreed it does make sense to have objects different from those stored in the database concerning their structure. But a table is not a class it’s still a collection of records a compound structure.

  6. Kmorwath Says:

    1) Datasets are an encapsulation of a database API. You may like it or not, but they are OO.
    2) OORM have advantages and disadvantages. Many databases are not designed for and by a single application. And their design may be outside your control.
    3) When performance are critical, well written SQL and database code (packages, stored procedures) outperforms OORM, which often risk to bring back the old “dBase” model where processing is made client side, not database side. Sure, a lot depends on the OORM engine, but I’ve yet to see one really good. That said, for applications that uses mostly a database as a “data dump”, OORM works.
    4) Data business logic should be in the database itself as much as possible. It ensure it’s coherent across applications, and changes are needed in one single point.
    5) Delphi really ties too much the UI to the data classes. It would need big changes there, because the 1995 model is now really outdated. Live bindings are not the solution.
    6) Database calls should have been always asynchronous since D2, but again Delphi did very little to help you with its strongly single-thread-desktop-app oriented model. Many “old” Delphi developers find very difficult to implement multithreaded apps, they lack the knowledge to implement them properly.

  7. Larry Hengen Says:

    @Leonardo,

    The beautiful thing about DevExpress grid controls is that the use a provider model so you can feed the grid data from just about any source, including a list of objects. That is why I support the DevExpress QuantuumGrid in hcOPF.

  8. Larry Hengen Says:

    @Stefan,

    You make some valid points, some of which are not news to me, but I don’t see how EMBT can push the product in the same way they did 20 years ago when the software development world has changed.

    We know better now, and all vendors should be encouraging modern best practices (at least providing some guidance). Otherwise, the problem is that the technology (Delphi) gets equated with the implementation (results) and develops a negative reputation for crappy results when in fact it is the architecture and the code of the of the application that is responsible.

    I would love to see a standard MVVM framework for Delphi ;-)

  9. Robert Horbury-Smith Says:

    We’ve kind of focused on two extremes here, but there’s a third option - which is the one I like.

    Option 1 - the VB approach.

    Some idiot cuts and pastes the same lump of code into the OnClick event of 200 buttons in the application, then subtly refactors each one. Then bungs 20 different datasets, accessing the same table, on 20 different forms - each dataset instance has subtly different bits of business and I/O logic in the event handlers. One day the application collapses in a screaming heap and no sane person is willing to have anything to do with it.

    Option 2 - the extreme OOP approach.

    Some dude hand crafts 300 or more classes for the 300 database tables, together with definitions for each table attribute as well as getters and setters, and then spends the next 20 years of his/her life refactoring the inevitable changes that take place in the database schema over time.

    Option 3 - (we haven’t mentioned this).

    Dude builds an architecture that as closely as possible represents the database schema by creating a group of RELATED DataModules containing “Dataset” (dbExpress, Firedac or whatever) assets that closely mirror the database’s logical design, and encaptulates the getters and setters and business logic within those DataModules. Those methods are called by the events (button clicks etc) in the GUI (guess that’s the “View”).

    Option 1 isn’t RAD - it’s just dumb.

    Option 2 - for me life is just too short.

    Option 3 ? That’s RAD, and satisfies many of the OOP requirements, and it’s the approach I like.

    By the way, anyone remember ECO (Enterprise Core Objects) in Delphi? How I wish we still had a solution like that (UML) in the Delphi domain (sob).

  10. Mike Margerum Says:

    I’ve spent a good deal of time studying MVVM and went through Malcolm Groves excellent video. This is a great way to building apps.

    However, there’s nothing wrong with using datasets. I have a 30 million dollar a year business running on a 2 tier ADO / SQL Server Delphi system and have had very little trouble extending it or maintaining it. Having a direct mapping of your relational model to your app domain model reduces a lot of friction and more importantly a lot of coding. Time is money and getting features out in a timely basis is more important than doing it the purest way.

    I’m in total agreement here that if you have the time an MVVM solution is the better approach. I just do not agree that you can’t build extensible systems using TDataset. FiredDac has some really great features built into their dataset that would take a heck of a lot of code to reproduce.

  11. Larry Hengen Says:

    @Mike,

    I would love to know what business you’re in that can generate $30 million a year from a Client/Server application.

    IMO, TDataSets still have a place in an application when it comes to data presentation and reporting. What I disagree with is using them for business logic. My experience has shown that unless you are very careful in crafting the implementation (as in @Robert’s option 3) you get an unmaintainable mess with a spiderweb of events being triggered. The reality is that most RAD based Delphi products are not developed with a great deal of architectural design and I have had to repeatedly deal with the result so I know what that can do to the financials of a company and the morale of it’s developers.

  12. Mike Margerum Says:

    Thats $30 million is sales,. not profit. Although the business is highly profitable.

    hehe and its not my profit i just do work for them.

    I built the Delphi App in 2004 and its been an absolute rock. Easy to maintain and extend. I love SQL Server.

    this 2 tier system does everything except for some mobile modules which I built for iOS that tie into SQL Server via some middleware.

    I’ve been in this industry for over 25 years. The buzzwords change and approaches keep getting recycled but the bottom line is a good programmer is going to write good code no matter what the tool.

    No matter how much test coverage you have a crappy programmer is going to still write crappy code. This is why i mostly work alone.

Leave a Reply