Tuesday, December 13, 2005
DSLs are not frameworks
If you consider domain specific modeling products as simply glorified code generation tools, you might get away with just rushing through the product documentation and jotting down the first thing that comes to your mind.
However, as vendors are becoming more successful in fulfilling the promise of enabling people within organizations to model their own domain specific languages, many are realizing that simply mastering the features (and limitations) of a particular modeling tool is manifestly insufficient.
I'm certainly not trying to classify such tools as interchangeable commodities; in fact, I speculate that, at least for the next few years, the choice of one tool versus another could easily contribute to the failure of a project.
There are already several documented cases of organizations that use modeling tools to drive entire product lines with enormous success. Frankly, I just assume that the very same tools have been employed within different realities with opposite results.
One of the goals for DSL authors is to generate as much code as possible whilst hiding implementation details behind reasonably flexible models and notations. This goal presents challenges that go beyond the tool in use.
If it is that hard, who should create DSLs? Will organizations need to create a specialized development role as some are arguing?
It is realistic to assume that experienced developers/architects who fully understand the domain and have been involved in the development of enterprise frameworks are the natural candidates for authoring well thought DSLs for an individual organization.
On the other hand, there is a danger that such authors might treat DSLs
exactly
like class frameworks, without realizing that the forces at work are significantly different.
When building frameworks we don’t necessarily know how our components will be consumed. For example, if we create a custom control and omit to expose a particular event, the entire control might turn out to be totally unsuitable for a whole category of users. It is then reasonable to take a “safe” approach by adding extra options, in the form of events, method overloads, extra parameters, extensibility patterns, etc.
However, using this tactic for DSLs would lead us to create “flexible” implementations of a language through the introduction of (superfluous)
configuration/customization
options to our models, without considering that every time we add an extra element, attribute or connection to a DSL we arguably make the language more difficult to use and, ironically, less adaptable:
- It is more difficult from the user perspective because there are more elements in the language to understand and deal with.
- It is less resilient to change because those options might expose subtle aspects of design and implementation of a solution that, over time, would be harder to modify and improve.
Of course, since we want to use DSLs to solve categories of problems, we cannot totally eliminate all forms of variation, but my point is that, in order to add flexibility to a domain specific language, we should take a systematic approach and
reduce options
rather than blindly increase them!
Is it possible? Yes, if we narrow the scope of application of the language to a well-known domain and we favor
abstractions
and
conventions
over
configuration options
.
Sunday, November 13, 2005
Today I'm finally breaking radio silence to express and share with you all my joy as
InnerWorkings
has been honored with the prestigious "New Company of the Year" software industry award by the Irish Software Association.
On Friday night I attended the formal ceremony together with four colleagues from the management team. Hundreds of people, including politicians and personalities from the software industry were rigorously dressed for the big occasion.
I was particularly nervous since the week before I was personally interviewed by one of the judging panels together with Fran (our CEO).
I already considered it a privilege to be short listed for both the "Technical Innovation" and the "New Company of the Year" awards, but given the names we were up against, I thought that our chances of winning were realistically slim.
At the risk of sounding clichéd, I want to give my personal congratulations to Fran and all my colleagues in US, Canada and Europe, who are delivering on a very audacious vision conceived less than three years ago. We still have lots to learn and improve, but InnerWorkings truly represents an unstoppable creative force and I'm proud to be part of it.
Finally, I want to express my sincere gratitude to the R&D team I lead and serve.
Guys, you are the most talented team I have ever worked with.
Thank you, thank you, thank you!
Monday, August 08, 2005
Last week Eamon O'Tuathail of
Clipcode
released a
Domain Specific Language (DSL)
for some of the
Gang of Four
design patterns using the
Microsoft DSL Tools for Visual Studio 2005
.
It is indeed a very remarkable proof of concept and although the implementation is still pretty rough, it represents a valuable model for discussion and learning.
Eamon is very active (particularly) within the Irish community and he is a regular speaker at the
Irish .Net Developer Alliance
(INDA) events where I often see him.
I briefly talked to him last night as we both were present at the excellent
Roy Osherove
's talk on agile development and unit testing best practices.
Eamon told me that he only spent a couple of days developing it; this is something I consider rather impressive given the premature stage of the supporting tools currently available (I bet that his superb documentation took him more time to develop ;-)).
The DSL designer currently allows setup and code generation for five patterns:
- Abstract Factory
- Builder
- Factory Method
- Prototype
- Singleton
Overall, I liked the organization of the code templates in the debug solution, as they are neatly located in a common folder and are referenced only through the ClipcodeEngineTransform.ReportTemplate file which acts as a
template controller
for all the instantiated designers.
Besides some understandable limitations of both the language and the underlying platform (i.e. the DSL Tools) that will be undoubtedly addressed in future releases, my main interest (and point of concern) lies in the level of abstraction of the model.
Using the language to generate code that realizes a pattern is pretty straightforward as the designer diligently constrains allowable classes and connections.
However, my worry is that, in order to realize a pattern, you need to select every single participant (class/object) from the toolbox and manually connect it to the permitted collaborator(s). As a consequence, you end up creating a diagram that matches quite closely the classic UML representation of each pattern. But do we really need to visualize the structure of a pattern?
As my exploration of the Domain-Specific Modeling (DSM) world progresses, I'm more and more obsessed by the idea that if we want to create the biggest ROI, it is essential to push our models towards a level of abstraction that is
significantly
higher than the level at which code statements and classes are written.
In other words, a
one-to-one mapping
between a class created in a diagram and a class generated in code does not generally save enough time and I would dare to consider it an
anti-pattern
.
A symptom is that the toolbox is generally filled with lots of different elements even if the main concepts (in this case the GoF patterns) are more coarse grained.
So here is a challenge that anyone interested can take. In order to realize an abstract factory, I should need to drag only one class on the designer (e.g. AbstractClassRealization) and setup its participants (AbstractFactory, ConcreteFactory, AbstractProduct, ConcreteProduct) using compartments only. Is it possible? I guess so. I will try soon anyways.
By the way, if you think about it, a GoF pattern is still at a pretty low level of abstraction. In fact, it is the very essence of an OO pattern to address recurring cross-domain concerns. So a "real" DSL may still be built on top of it and possibly provide an even higher return of investment for your organization.
Sunday, July 10, 2005
My good friend
Marcus Mac Innes
is just back from Amsterdam and posted a lucid
report
of what he saw at Tech-Ed Europe.
His apartment faces our offices in Dublin, so it is natural for us to occasionally meet and share some ideas.
I can't wait to have a chat with him again over a good cup of (Italian :-)) coffee!
With Marcus I often talk about many topics, but lately SOA dominates a lot of our conversations.
I'm actually very surprised to read that about 60% of the audience
claimed
to be currently building SOA based applications.
I suspect that a lot of them are simply creating basic
web service façades
over their existing applications.
That is so !SOA.
Sunday, July 03, 2005
Tech-Ed Europe is about to start and I join
Bill O'Brian
in the multitude of really envious people that would love to be there.
Instead I'm trapped here in Dublin, craving for news from Amsterdam.
I don't have any reasons to complain, really. After all, I was in Orlando just a few weeks ago.
In fact, I consider this coming event as an occasion to go through my
mind maps
and, once again, reflect on my recent experiences.
Bill recommended Gregor Hohpe's session on event-driven architectures.
I cannot agree with him more.
Gregor is a true expert on asynchronous messaging patterns and co-wrote what I consider the best book on this subject (
Enterprise Integration Patterns
).
By the way, if you believe that asynchronous messaging technologies such as Message Queuing (MSMQ) are not for you, I actually
implore
you to go to one of Gregor's sessions.
Maybe you will think about it next time you will wake up early trying to book tickets for your next U2 concert and the site goes down because
suddenly
there are too many requests that the server can't cope with ;-)
In addition to Bill's
favorites
, I would include Ron Jacobs's "ARC313 - Patterns for SOA". The patterns and anti-patterns described in that session are simply invaluable and I'll be certainly sharing some thoughts about them in the future.
Finally, I certainly suggest spending some quality time at the so-called Chalk-&-Talk/Cabana sessions. It is such an excellent way to meet a lot of talented people and share insightful ideas and tips!
Sunday, June 19, 2005
About eleven years ago, I got involved in the development of my first "serious" client-server application using Visual Basic 3.0 and Oracle (using a data access API called Oracle Objects for OLE).
We were a small team of 3 developers working together for several months and, to be honest, I don't think we actually knew what we were doing (well, I'm sure I didn't! :-)).
I really enjoyed those times and it certainly was a valuable experience.
Back then, I had the pleasure to work with Manlio R.
Manlio was essentially what I consider a pragmatic anti-Microsoft guy; someone who fundamentally dislikes Microsoft but earns a good living by using Microsoft technologies.
He was so funny though. Whenever we would find a bug in Visual Basic he would smile and say something like "Microsoft is only good at building nice dialog boxes; Oracle/Linux is the way to go!"
It didn't matter if our bookshelf was full of Oracle manuals as the software wasn't that user-friendly or bug-free either.
Even WinWord would not pass lightly his severe judgment as the VI editor was a way better tool in his hands.
I haven't seen Manlio since then, and we only stayed in touch with the occasional emails in the first few years that followed. Then I totally lost contact.
Manlio now has his own family and he is an accomplished J2EE/Agile developer.
How do I know? He surprisingly found one of my posts and, after so many years, he dropped me an email to say hello.
Thanks Manlio, you are the man!
Saturday, June 18, 2005
“SOA is an architectural approach to creating systems built from autonomous services.” — Ron Jacobs, Tech-Ed 2005
While this is certainly true, it does not fully explain why so many “experts” and practitioners debate and, sometimes, disagree about what constitutes a well factored SOA-based distributed system.
For some strange reason, SOA reminds me of Kent Beck's description of how he came to devise the Extreme Programming methodology.
Kent had the mental picture of a control panel with knobs; each knob would represent a practice that, from his experience, he knew worked well.
He then imagined turning those knobs all up to 10 to see the consequences.
The same analogy helps me to capture what I believe is the true spirit of SOA.
I see it as being an architectural model that mainly limits to the extreme the
assumptions
we make about the services that we create and consume in terms of reliability, communication protocols, network latency, implementation platforms, resiliency, versioning, security, coupling, etc.
I like to think that people around the world are simply attempting to define what "turning a knob to 10" means, evaluating implications, and eventually devising practices that work.
This oversimplification helps me to understand why, for example, some experts suggest that message exchange between services should be fundamentally asynchronous; synchronicity would mean taking for granted certain assumptions about the responsiveness of the services we consume which for some are simply unacceptable.
Friday, June 10, 2005
Wednesday’s session about the DSL tools for Model-Driven Development in VS2005 was truly fascinating. Actually, I think that it justifies the investment of my trip across the Atlantic alone!
After illustrating that domain specific languages are ways to improve effectiveness in communication,
Jochen Seemann
literally blew my mind by demonstrating what can be done today with the CTP release version of the upcoming
Microsoft Domain-Specific Language (DSL) Tools
.
This SDK allows you to
easily
build a graphical designer hosted by Visual Studio 2005 that is able to provide support for a custom domain-specific language.
What does it mean? Once the designer is built, a user can instantiate the designer, drag custom shapes from the toolbox, connect those shapes together (with validation support according to the language rules) and use the template engine to generate code/artifacts!
Building a DSL designer requires several steps, which basically involve the definition of:
- the elements of our domain in a domain model (classes and relationships)
- the notational elements (shapes and connectors) for the business entities
- the legal rules (validation/constraints) from a notational perspective, and the mapping of the notation to the elements of our domain.
Once the designer is built and instantiated we can use template-based generators to generate artifacts from our model (e.g. code, HTML, XML, WordML, etc). The templates have a notation similar to CodeSmith or classic ASP, and in the future some form of Intellisense (to query the model) is likely to be provided.
Some of these steps currently require the editing of an XML file. However Jochen said that this is really an “alpha” version of the toolkit and that better support will be provided in further iterations (indeed, the textual generator engine, code named T4, is only 6 weeks old!).
He plans to release incremental versions of the toolkit every 2-3 months, maybe even after version 1.0 will be shipped.
Currently I have only a few random observations/considerations on DSL designers that I want to dump here for my reference and eventually refine in future posts as I become more experienced in this field:
- We can build multiple designers to generate different parts in our systems, so we can (and probably should) start small.
- 2-Way synchronization (e.g. graphical model – to generated code – back to graphical model) is probably not going to be provided with version 1.0. It is a difficult problem to solve and I agree with Jochen's pragmatism that it would be a fundamental feature only in a relatively small number of cases.
- Template/designer versioning and deployment are going to be difficult issues. We will need a defined structure and guidelines to avoid dealing with a total mess.
- A DSL designer is likely to work effectively in areas when the domain entities are very well understood. I can see how the use of a ubiquitous language would be core in this context.
- DSL designers could be used for requirements; however I have some reservation in this direction as the domain model is likely to be refined and changed way too quickly in a real world scenario. Use cases, CRC cards, and UML drafts are still likely to be my preferred option in the short term. I might be totally wrong on this, and I hope somebody can change my ideas soon (any volunteers?).
- There is a space for graphic tools and a space for code. A guy asked if Microsoft was going to provide basic notational elements that would give us very fine grained control capabilities (ifs, whiles etc). I think that writing code using shapes is insane. This is not a new idea (companies have been trying for years to propose solutions in this direction) but when you get pages and pages of icons you quickly realize that you are in the wrong space.
- Code generation driven by a designer is very cool. However writing templates is a pain and it feels like going back to messy scripting languages.
- Aspect Orientation could have a major role in this type of development. You add an attribute to a shape and (with the right template) your code magically gets to be generated. Jochen is not totally convinced by the feasibility of the AO approach as many people got burned. After talking to the guys in Avanade, however, I’ll be willing to take a serious shot in my problem space.
- It will be fundamental to maintain a clear separation between generated code and custom code. Use of partial classes that can be regenerated would be an option, separate classes and aggregation/inheritance another.
-
I wish we could have several injection points in the designer and even a link to MSBuild, so that separate tasks could be executed upon code generation.
I hope this post gave you a sense of what this technology can provide. As I said at the beginning, I’ve been totally fascinated by it.
Finally let me just say that this technology gave me a fantastic idea that promises to provide an incredibly elegant solution to a huge problem in
Innerworkings
’ domain. Unfortunately I’m only teasing you guys because I can’t tell you what it is, as probably Fran (my CEO) would literally kill me if I did :-)
Wednesday, June 08, 2005
After the code generation session, I attended Roger Sessions’ talk about the next generation of SOAs. I thought that the speech was very interesting as Roger described the challenges that companies will face in the next 3 years. He sees three main challenges: technical, regulatory and competitive. From the technical point of view, it’s clear that the standards are moving from the basic point-to-point Soap model (where a message is “simply” encapsulated in an envelope) to the point-to-intermediary-to-intermediary-to-point WS-* model, where headers are used more heavily and limit the body of the message substantially. Roger calls this model WS-SCRAM : Secure, Coordinated, Reliable, Asynchronous Messaging and went into the details of describing each aspect. Nothing really new to me, but he also illustrated how to model collaborations within software systems describing the concept of a Software Fortress. He actually wrote a book about it so I will definitely investigate further.
One serious aspect that is often ignored in the field is the necessity for systems to comply with ever increasing and stringent regulations. This is something that us Italians are very familiar with (well,
Italy
is very strict with data protection laws). Essentially, in several business fields, data has to be traceable, auditable and lockable. Fail to do that and your CEO could go to jail…actually, that might not be my problem after all, what do you think Fran :-)?
In the afternoon I went to other talks. I was particularly interested in the “Design Considerations for Enterprise Application Frameworks” session given by Kyle Huntley of Avanade. He heavily collaborated to the development of the “famous” patterns & practices Enterprise Library.
Kyle classified frameworks in two main categories: helper frameworks (for example, the Logging and Instrumentation Application Block) and structural frameworks (for example the User Interface Process Application Block). I found this distinction particularly helpful as there are very distinct benefits and issues. Usually helpers are easy to consume, can evolve with relatively low impact and they usually have low resistance in their adoption among developers. Structural frameworks have usually a much greater impact, they have a higher learning curve, and their adoption have higher costs, depending on the type of organization. There were a lot of insightful ideas and “tricks” that I will certainly consider once I’ll be back in the office. In my organization we work really hard to continuously enhance and improve our internal framework, so I was very familiar with the issues and practices that Kyle illustrated.
I actually met Jacob Croft (also of Avanade) a couple of hours before at the Innerworkings booth and, among other things, he managed to get me interested again in Aspect Orientation.
I know, it is a topic that often raises doubts as it is very hard to design aspects properly in order to avoid interdependencies. Jacob is pretty confident that with a bit of lateral thinking you can really obtain huge gains, and he confirmed that Avanade has been really successful in its implementation within their internal ACA.NET framework.
By the way, if you haven’t used the Enterprise Library yet (and you develop non trivial applications) stop reading my post and download it
now
!
After other talks, I ended the evening going to (not one but) two parties (VSIP and MSDN)! Wow, I really drank too much yesterday, but I had so much fun!
My second day at Tech-Ed has been pretty intense.
After the announcement of the expected release date for Visual Studio/SQL Server 2005 and BizTalk 2006 by the week starting Nov 7 in Paul Flessener’s keynote, I went to a fairly interesting session about code generation. I’m not too sure I agree with all the implementation details exposed by the speaker, but it was definitely worth listening. The scenario involved creating a sophisticated façade on top of a big application written years before in C++. The legacy application implemented a rather poorly written interface which included late bound calls and string concatenations. Building this façade involved creating a DSL (Domain Specific Language…just in case you live in another planet and you haven’t heard about it yet :-) and using it as a starting point for code generation. The point was to use this approach with today’s technologies without having to wait for the sophisticated designers provided by, say, Visual Studio .NET 2005. So, he pretty much used XML schemas and WSDL files to define the language and operations and then used code generation tools (CodeSmith, XSD.exe and others) to generate assemblies, documentation etc. I found interesting that he used WSDL even if web services weren’t involved (effectively, you only need to replace the bindings).
In general, this year I found that there is a disproportional push for code generation, templates and DSL. It kind of weird as the concept is definitely not new.
I will write about the other past sessions later on today as I must run now.
Tuesday, June 07, 2005
After a long (and troubled) flight from
Dublin
, I finally arrived to
Orlando
last night, ready to participate to Tech-Ed FL. Cool eh? InnerWorkings is also involved as exhibitor, which means that I get to talk to lots of developers whenever I get a chance to hang around the booth. I’m now towards the end of the first day and I’m just wrapping up before going to Sea World tonight for some party (eh eh, are you jealous?…well you should!).
How has been the day so far? Well I’m pretty sure that you already got the few announcements that Steve Ballmer made this morning in his keynote. The speech was centered on Microsoft helping developers and IT pros to work together, to connect people and information independently from their location. So we’ve been tortured with a list of Microsoft products and marketing info…Bah, and I haven’t slept in 24 hours “to be part of this”? Maybe I was sleeping too much, but it looks to me that the only announcement of “some” interest to developers was the Visual Studio Tools for Office support for Outlook add-ins. In reality it’s my fault, I’m just not a fan of keynotes…actually, I hate keynotes. I especially hate them when they are very product centric, regardless on who the speaker is.
Afterwards, I went to the Architecture Certification Chalk Talk which turned out to be a very instructive Q&A session about the upcoming program that aims at creating an elite group of highly regarded professionals. Our group was very small, there were no more than 10 of us asking questions. Gregor Hohpe was among us. Unfortunately I didn’t have a chance to talk to him as he had to run away. Lately, I have been discussing the finer details of a particular messaging pattern with a collegue of mine, and I hoped that Gregor could clear some our doubts. Sorry Jeno, I will try to talk to him later this week!
I also went to the “Building and Using a Software Factory” session. This is something that is definitely in my radar as currently we have about 400 solutions at InnerWorkings and we would certainly provide a good platform for automation.
Oops, it’s getting late…I must go to the party!!
Monday, May 23, 2005
In this
post
, Jimmy Nilsson observes that using a ubiquitous language poses a problem when working in a business based in a country where English is not the native language. Customers typically use their local language, but he
prefers
writing software in English.
My view is that programming in English is not a matter of preference only.
Before moving to
Ireland
about 5 years ago, I worked for a few companies in
Italy
, including a very successful firm in the field of software automation. Using Italian names for classes and operations seemed like a good idea at the time…until, a few years later Siemens A&G bought us!
Sunday, May 22, 2005
In the book titled "
Domain-Driven Design
", Eric Evans suggests that one of the fundamental ingredients of effective software modeling is the deliberate adoption of a shared language between domain experts and software developers.
Such a
ubiquitous language
develops as the understanding of the domain evolves.
The idea is that by eliminating the linguistic barriers between different groups, we enable the creation and refinement of a rich
model
that embeds the language in code.
This model should be sufficiently expressive to support both domain experts and developers in their communication of rules, requirements, etc.
I'm still not sure about the feasibility of such approach, but don't get me wrong, the objective is certainly admirable.
My fear is that our capability of building a ubiquitous language on which our model should be based is inversely proportional to the number of individuals and groups involved in the effort.
Each member tends to have only a partial view of the domain (including domain experts!). Each individual has different mental maps about how things are or should be.
The process of creating and sharing both language and underlying model could face challenges similar to the ones that experts find in an enterprise when adopting a shared database strategy or a global canonical model: it’s very hard to do properly (trust me…I know.).
Are there valid alternatives? I don't know for sure. Use cases, for example, are great at bridging the gap between domain experts, users and developers. But admittedly they are not of huge help in all situations and they give limited or no support for validating models and business rules.
Whatever style we adopt for gathering requirements however, it is important to keep the right attitude, and learn the most important skill of all: the ability to
listen
.
The biggest failures in software development often happen when developers or analysts don't question their mental maps enough to ensure that those accurately reflect reality.
Sunday, November 07, 2004
A couple of years ago I participated to a very interesting summit for .NET architects in the Microsoft Campus in Redmond. At that time the
PAG group
just started to deliver a couple of application blocks and the idea of developing a pattern-aware .NET community was at its infancy.
Personally, I felt that the community was (and still is), by a large degree, almost totally unaware of even the most basic design patterns. That was in shear contrast with the Java world, where every solution to recurring problems was identified, shared, evaluated, catalogued and then framed as either a pattern or an anti-pattern. If you followed the "PetShop war" or you checked
theserverside.com
and looked at the quality of the posts you would know what I mean.
Anyways, in that occasion, I personally met
Gregor Hohpe
from ThoughtWorks. Gregor had just finished co-writing the book "
Enterprise Solution Patterns Using Microsoft .NET
" and I had the opportunity (like many others) to have a look at his manuscript just days before its release.
As we obviously shared a passion for patterns and architectures, we talked about things that we both liked and briefly shared some experiences.
I was really curious to know how he developed the necessary skills to be able to write such a book. With modesty, he said he was lucky because he had an early involvement in a group of very smart and influential people. Additionally, a lot of those patterns where adapted from existing implementations, mostly in the Java platform, on which he had a strong background.
When I asked Gregor what kind of methodologies he uses to develop enterprise applications, he told me something that was going to transform my career forever: he was an agile developer.
WHAT????? I was really shocked. How could that be possible? With all my ignorance, I thought that agile methodologies such as Extreme Programming were yet another RAD fad, an excuse to masquerade hacked development shortcuts with more or less indifference for consolidated design practices.
Evidently, I was wrong.
So, I questioned my beliefs and embraced change.
Two years later, I'm sitting here and writing my first post ever.
My mission is to share my personal and professional transformations with you.