Today, as we write application code
we generally are not dealing with immutable state/data, we are applying application logic to state/data so that it transforms from one known state to another known state in a consistent, all or nothing fashion. Today, when performing this in code that does not involve Transaction Managers or Resource Managers, we use Try/Catch exception handling statements to mimick the "Commit" and "Rollback" concepts of transactions. The difficulties with such an approach is that as we scope certain types of work and/or scope more than a few simple actions into a unit of work, the level of complexity in nested try/catch code grows and can easy reach a point where it cannot truely provide the "Commit" and "Rollback" concepts of a Transaction.
For example,
take the Try/Catch below
try
{
// operation(stateX)
// operation(stateY)
// operation(stateZ)
}
catch
{
// cleanup after states X, Y, and/or Z, as something has gone wrong and we need to get to a previously known state
}
The challenges with using try/catch in a Commit/Rollback style are some of the following:
-
If stateX is shared between between more that one execution contexts, another execution context may read an inconsistent state and hence a "dirty" read, as the scoped unit of work may fail
- What if an operation on stateY deletes/alters a resource and how do we "Rollback" in the catch.
- How do we coordinate with resources that are transactional, i.e. if stateX represents a call to a transaction queuing/messaging or database system. How we coodinate between the different resources and unit of works!
- and there is lots of other problems
While all of the above can be eventually handled in code, with careful planning and structure, but what is the essence of what we are trying to achieve in the code and does it form a pattern! It is the "Commit" and "Rollback" concepts of a Transaction and if we factor the patterns, interfaces, contracts and interactions of Transacional Processing into the code we have more flexibility and agility. Please note I am not suggesting that every try/catch has a "Commit" or "Rollback" focus, but we sometimes use try/catch to group actions into a unit of work in which we determine the outcome of a unit of work by interrogation (using if statements) and do not negotiate/coordinate the outcome of the unit of work (as an example). Also, to provide the most flexibility and agility, we need for the actions that are grouped into a unit of work to be able to be aware of the (transactional) context. We also have the problem that try/catch is not context aware, i.e. it only knows a single execution context, and while dealing with a single execution context may suit us some times, we need to be able to handle both concurrency management and recovery control at the same time!
One approach to providing a cleaner implementation, is to factor out the interrogation and make it the negotiation/coordination of the outcome from the application logic and for an intermediary to work as a coordinator between the resources being used and the application logic. The coordinator controls the enlistment of resources within a scoped execution context (as defined by the application code) that we want to be transactional. We then need to define a number of common interfaces and interaction semantics, so that the use/enlistment of one or more resources in the scope/context of a unit of work is clearly defined. This can be achieved by having methods/semantics/behaviors common to all resources so that the different stages of moving/transforming from one known state to another known state as a unit of work or as a part of a unit of work, happens in an all or nothing fashion.
All of this fits quite nicely with the 2 phase commit (and even the one phase commit) protocol as to how the "Commit" or "Rollback" semantics of units of work can be achieved.
Oppps ....... Hmmmm ..... since I thought I was going to be able to do this is 3 to 4 postings, I seem to have underestimated and need one or two more. I think I will leave it here for now with this posting and follow up with a part 4,5,6
![]()
In general, when we think of Transactions we tend to think of database transactions. Databases are a great example of how a data server can implement transactions and therefore handle concurrency, provide isolation and provide consistency. The concept of what constitutes a Transaction expands beyond that of a Database Transaction and with this we have more and more areas where the abstraction and contracts that make up a Transaction can help us with the problem of how to handle concurrency management and recovery, as we apply application logic to state and consistently move from one known state to another.
So when we look at a Transaction, what are main parts involved? At a base level it is composed of three parties, i.e. an Application, a resource manager and a transaction manager. So how they are involved in a transaction and what is their role in a transaction?
The Application is pretty self explanatory, it’s your code. Its role in a transaction is to define the scope of a transaction, perform read/write operations on a resource manger and specify whether it views the state as being consistent. A resource manager is a component that manages a stable data storage system (i.e. it always shows a consistent view of data/state), and participates in the two phase commit and recovery protocols with the transaction manager. A transaction manager tracks which resource managers participate in the transaction, provides an interface to the application – start, commit, rollback and also, coordinates the transaction outcome between the resource managers. The transaction manager has the responsibility of properly handling error conditions, on behalf of an application, recover from them and ensure that the resource managers are always in a consistent state.
With today’s technologies the Resource Managers that typically get used are those of databases systems and queue managers. But is there other times when you need to be transforming/moving from one state to another? There certainly are!
The basic premise of Transactions is the coordination/negotation of an outcome that groups actions into a unit of work that transforms from one known state to another. As the transformation of state, the coordination/negotation of an outcome, or the grouping of action into a unit of work are something we do very often in application logic, is this not a Transaction.
So when do we get involved in the transformation of state, the coordination/negotation of an outcome, or the grouping of action(s) into a unit of work? What are we trying to achieve, how do we generally do it today and is there a better way to do it?
I've kicked off Part 3, to share my views on this and hopefully provide some insight!
![]()
When thinking about Transactions, I think it is fair to say that Transactions are one of the most powerful and frequently used paradigms in Enterprise Computing today. With Transactions, we have a collection of patterns that deal with how concurrency management and recovery control can be handled in possible shared/concurrent environments/contexts and it is this that ensures data/state correctness. It is the key concepts and abstractions of Transactions maintaining data/state consistency in possible concurrent state/data access environments (where failures may occur) that have allowed us as developers and architects to be largely released from having to deal with the tricky aspect of maintaining the correctness of state in concurrent environments in an enterprise application.
Two of the very interesting aspects of Transactions is how trusted they have become and how we perceive them. Concurrency Management and recovery control are some of the more tricky aspects of Enterprise Computing, but yet when leveraging the Transactional paradigm and executing units of work in a concurrent environment in the context of a transaction, we are not overly concerned about isolation or failure as much of the work is done for us by the Transaction Paradigm. This is a testament to the level of trust and adoption that Transactions have achieved as an abstraction and how it has evolved into a framework of runtime services that takes most of the responsibility for concurrency management and recovery control from us as developers and architects. The paradigm of Transactions can be seen as a set of patterns, Interfaces and Contracts that can be leveraged to simplify the transformation from one known state to another and without these Interfaces and Contracts; a lot of code, time and effort has to be spend in making sure that data/state transformations happen in a consistent and all or nothing fashion.
It is these key abstractions in the form of runtime services or frameworks that have provided great productive gains for us as developers and architects, as it allows us to focus on business logic and allows for the underlying runtime services to deal with concurrency management and recovery control. Therefore as we implement application logic, by applying units of work to state, we want the transformation from one state to another to be applied in a consistent (all or nothing) fashion, as well as wanting to be isolated from other parallel activities. As I mentioned in my
How should we think of Transactions ?
blog posting, I like to think of the Transaction paradigm as a collection of patterns, interfaces and contracts that can be leveraged to tackle the issue of data/state correctness as we group actions into unit of works to transform from one known state to another. Some of this collection of patterns, interfaces and contracts are short running transactions that in the main adhere to the ACID properties but there is a rich portfolio of transactional types that we do not focus on that much and/or maybe perceive as not meeting the ACID properties (see
How should we think of Transactions ?
and
Volatile Transactions & In-Memory Transactions - do they have ACID, ACId or ACI properties?
)
Today, the most common example of how the Transaction paradigm can provide concurrency management and recovery control is Database Management Systems (DBMS). Databases (aka DBMS) are themselves another key and often used architectural piece of Enterprise computing. Database Management Systems (DBMS) provide a paramount role in how an Enterprise may store, retrieve and share state/data. But we sometimes draw an association between Databases and Transactions that is so close that they are often perceived as being interchangeable, i.e. Transactions are DB Transactions and nothing else. The close associations between databases systems and transactional processing have been fundamental in bringing the concept of Transactional Processing to the fore, but in my honest opinion I think that the close association may have also brought a narrowing of perception of what are Transactions and Transactional Processing.
I have put together a series of blog postings to share my current views/thoughts! After talking to a few folks about Transactions, it is very interesting to hear the different views and I'm not sure if there is a one size fits all for what is a transaction, but this is the thing about a paradigm and a snappy acronym, it can have a life of its own
Maybe we interchange the paradigm, acronym’s and implementations into one fixed view and its this fixed view that maybe causing the issue!
![]()
In my last blog posting,
How should we think of Transactions ?
I mentioned Volatile transactions and how I currently think they can be perceived in different ways! Do they have ACID, ACId or ACI properties?
In case you do not know what Volatile Transactions (aka In-Memory Transactions) are, they are transactions that do not persist to a durable store such as a hard disk (ouch! we have multiple definitions of durable to content with
). They are transactions that are done completely in memory and do not incur the I/O cost of writing to a hard disk a Transaction log or the actual known data/state.
What is interesting about Volatile transactions is that they are sometimes perceived as not being Transactional, because they are viewed as not meeting the ACID properties, especially the "D" property. I suspect the main reason for this view is because there are no Transaction logs and/or no data stored to hard disk.
So when you look at a typical definition of the ACID properties (like what I have listed below), some of the reasons why this view maybe held become apparent:
(NB. this is an amalgamation/synopsis of many definitions I have seen)
A
- Atomicity - Transactions are atomic (all or nothing)
C
- Consistency - The transaction is a correct transformation of state. Consistency constraints that are defined on the underlying data servers are preserved by a transaction.
I
- Isolation - Each transaction should appear to execute independently of other transactions that may be executing concurrently in the same environment.
D
- Durability - Once a transaction commits, it’s updates survive even, if the system goes down
ok, so a Volatile Transaction meets the ACI of the ACID properties, but not the D as defined above. But is this ("Once a transaction commits, it’s updates survive even, if the system goes down") a true reflection of what Durability means? Many similar variants of this definition gets used on a frequent based and they bring specifics to the definiton that are not about the failure of meeting the ACI properties or making the "Commited" value available/public but focus on the failure of media or a system. In my opinion the failure of media or a system is not an aspect of the paradigm or indeed part of the ACID properties, but an aspect of how the paradigm can be implemented and/or a requirement of the longevity of the state being persisted/determined. It is perhaps the fact that the best known implementation of the data server feature of the transactional paradigm is a database that has led us to shape the ACID properties to the specifics of databases. But perhaps the Transactional paradigm has more to offer than the correctness of data/state transformations on a database. The transaction paradigm has a core focus on simpilifing how we can achieve the correctness of data/state transformations where we may have to deal with concurrency management and recovery control and this can be applied to any transformation of state.
Perhaps a different way to look at the durability of ACID is where the definition of ACID as an acronym came from and how some of the leading experts in the industry define them. As far as I know the ACID acronym was first coined by Haerder and Reuter (1983). The definition I like to use is that by Gray and Reuter (1993). It is the following:
A
- Atomicity - A transaction's changes to state are atomic: either all happens or none happen. These changes include database changes, messages, and all actions on transducers.
C
- Consistency - A transaction is a correct transformation of the state. The actions taken as a group do not violate any of the integrity constraints associated with the state. This requires that the transaction be a correct program.
I
- Isolation - Even though transactions execute concurrently, it appears to each transaction, T, that others executed either before T or after T, but not both
D
- Durability - Once a transaction completes successfully (commits), its changes to the state survive failures.
When you read this definition of Durability (and the ACID properties overall), it in my option explains the feature richness of transactions and how it can be applied to the persistence of the transformation of state and also how it applies to the other properties such as A, C and I. The state survives failures by the data server ensuring that when the transaction commits a new durable state is persisted/stable/available/public and if something goes wrong (such as not be able to ensure Consistency) that the data server returns to the previous known state. In essence, this is a contract that guarantees the persistence and stability of the state and that the outcome of a transaction is known/public and usable in subsequent transactions, as a known/correct state. A good definition I have seen is that once a transaction is committed, it cannot be abrogated.
With Durability being about the persistence, stability and making public of “commited”/known state as we transform from one known/correct state to another, it allows us to leverage the transaction paradigm in many scenarios. For example when using Volatile Transactions, we are leveraging data servers that store data on Volatile media, such as RAM and providing the transaction paradigm in-memory. With the common misunderstanding that Transactions are exclusively for databases, the question arises about the “D” property.
For me the Durability aspect of a transaction is present in a Volatile transaction. The question is whether its “D” or “d”. Since most transactions are based on the X/Open Transaction processing model, the requirement for log managers or lock managers in resource managers (data servers) are not central or core to implementing the transactional paradigm at a base level. In relation to the usage of Volatile media for Transactions? in my opinion the requirement of the media used by the data server is that it is able to present a stable/persistent/public state once "Commit" has occured.
So I believe that Volatile Transactions meet all of the ACID properties and that its ACID with a "D"
phew! long post.
I guess the crux of the issue is the 2 uses of "durable" when talking about Transactions. One is the "D" in the ACID properties and the other is the property of the data/state/media being used (and how it is stored). The "D" in the ACID properties is more about making a "commited" value available and the other is about an implementation requirement of how long (and/or resilient) a known/committed state must be.
![]()
So now the news is out, there is going to be a PDC 2005
There are currently no more details available
, but you can
signup
for updates as the details develop.
![]()
Debunking a common misconception
There is a lot of confusion around the DefaultValue attribute - it feels like it really should setup the initial value of the property. Unfortunately, this is not the case. It’s used to determine whether the property should be written out to InitializeComponent, and what to reset the property to when someone right clicks in the property grid and selects “Reset”.
Continuing on
from last time
with the square control, adding a new property
Perhaps we don’t *always* want the hover rect to paint on mouse over.
We can give people the option of having that on by adding another property called “Popup”.
private
bool
popup =
false
;
public
bool
Popup {
get
{
return
popup;
}
set
{
popup =
value
;
}
}
In OnPaint we’ll check this property to see whether or not we should actually paint the rectangle.
When the square control is dropped onto the design surface, the following lines of code are generated:
this
.square1.Location =
new
System.Drawing.Point(96, 80);
this
.square1.Name = "square1";
this
.square1.Popup =
false
;
this
.square1.TabIndex = 2;
this
.square1.Text = "square1";
While this is correct, we don’t actually want initialize component to call the Popup method, because it’s not going to change anything… one could imagine perf problems if setting this back in was a slow operation.
Common Misconception:
Specifying the DefaultValue does NOT set the value of your property – it’s used to determine whether the property should be written out to InitializeComponent, and what to reset the property to when someone right clicks in the property grid and selects “Reset”.
|
If we add the
DefaultValueAttribute
to the top of the property declaration, we can essentially inform the serializer what the untouched value of the property would be (if no one ever played with it).
The serializer will then skip over properties that haven’t changed.
[DefaultValue(false)]
public
bool
Popup {
get
{
return
popup;
}
set
{
popup =
value
;
}
}
This is an alternative to using the
ShouldSerialize and Reset method pattern
.
If you wanted to do this using ShouldSerialize and Reset you would add these two functions to the square class:
private
bool
ShouldSerializePopup() {
return
Popup !=
false
;
}
private
void
ResetPopup() {
Popup =
false
;
}
Note these methods do not need to be public, as the serializer finds these methods via reflection.
The Reset method we have not talked about yet – this is what gets called when someone right clicks on the property in the PropertyGrid and chooses the “Reset” context menu item.
In general prefer DefaultValue to the ShouldSerialize method.
Ragha posted an article about
how the CodeDomSerializer figures all this out
a while back.
![]()
Andrew
- yet another windows forms groupie - has just joined the world of the blogging. Can't wait to see what he's got to say.
![]()
When someone brings up
Avalon
, I inevitably gush about how it will transform advanced desktop visualization from a dark art into something very accessible. I’m surely most like an evangelist when I get carried away about the creativity Avalon will unleash.
If you work in interface design or information visualization, you've probably come across
Edware Tufte
. His books
The Visual Display of Quantitative Information
and
Envisioning Information
should be on every reference shelf. Through Chris Sells (who had the good fortune of
spending a day with Tufte
) comes the story of Sean Gerety's
Avalon implementation of Sparklines
, which is one of Tufte’s
many elegant data visualization concepts
.
In the spirit of Tufte's clutter-free, elegant design, let me offer a couple of other things I’d love to see implemented in Avalon.
-
Web browsing that uses 3D to move beyond the forward-back metaphor. Think about it. How would that look?
-
The Tablet PC (pen-gesture-based) equivalent of the interface from Minority Report. (Aside: I love
my Tablet PC
. After events, it often turns into a communal sketchpad at the pub. I was thinking of posting up some of what gets sketched on it!)
![]()
I was a last-minute addition to
a panel discussion last night
in
Dublin’s Digital Hub
about the future of Game Development in Ireland.
Ireland is a country where we have a publisher (Microsoft), some
very strong middleware
, but almost no international presence on the game development scene, unlike most of our neighbours: England, Scotland and France are all well represented. So what’s the story?
I know firsthand that the necessary creative and technical talent exists in abundance here. We have
community interest
and
academic programs
cultivating internal talent. But
the games business is a business
, and Ireland lacks experience (and potentially core knowledge) pertaining to dev shop management - and particularly interaction with publishers. In a hits-driven industry, this is an essential skill. So there is a case for tax incentives etc to attract this knowledge into the country.
Mobile gaming
offers strong potential for a hit. Production costs are lower, the potential for innovation is arguably greater, and tools (e.g. the
.Net Compact Framework
) make it easier for smaller shops to concentrate on innovative ideas and content, and deploy games across multiple device platforms.
Managed DirectX
is another tool that makes it a lot easier to get demos (or full games!) together for the PC platform. ManagedDX makes good on the stat “98% of the speed for 50% of the code” – and again, this is
firsthand, 60fps knowledge
. If you don’t have a striking demo and you’re arranging to meet with a publisher, you might as well save yourself the plane ticket. So tools like the Compact Framework and Managed DirectX significantly lower the barrier of entry.
After last night's event, some local developers told me that they generally appreciated my point that trying to be “the next anything” is the kiss of death, and that in order to establish a Game Dev “beachhead” here in Ireland, we’ll need to start with a relatively inexpensive and highly innovative “Blair Witch” type hit. (Which I happen to believe will happen in the mobile space.)
The big unasked question was “do we want to encourage an Irish Game Dev industry?” Only a tiny fraction of the titles in production actually ever make it to the shelves. Do we really want to be encouraging such a volatile industry in a relatively small economy? Think of what happens to the folks pouring their hearts and souls into the titles that never ship!
But that… that’s the subject of another discussion.
![]()
Last February we had a great Forum event for Architects involving a full day of architectural content with excellent industry and Microsoft speakers.
This event will be back in Dublin on January 27th. The topic this year is interoperability and integration. There will be great speakers and this is an excellent way to meet other architects from across Ireland. If you are an architect and interested in attending please register here.
http://msevents-eu.microsoft.com/CUI/EventDetail.aspx?EventID=118753459&Culture=en-IE
Bill
![]()
Theres a potential XSS attack to watch for when allowing entry of Color values.
![]()
I’d like to thank
Sean Chase
for his comments on the
SLAR
up on Amazon..
As you saw
from my previous post
, I am in the middle of finishing up vol2, so now is a great time to get good and constructive feedback on Vol1.
Please feel free to drop
me a line
or, if you like
post a comment to Amazon
on the book so others can benefit as well.
BTW – thanks to all of you that responded to
my ask for reviewers
.
I am going to get back to everyone this weekend to get the ball rolling.
A Must-Have for .NET Developers
, December 8, 2004
This book is extremely insightful and interesting. Not only is there great information and example code (both within the book and on CD), you get to read comments from the designers of the BCL object model. Comments from the designers include: how to use, how not to use, what could have been implemented better, and other useful information. I was extremely impressed and the only downside is that now I'm left with wanting more volumes! Hopefully more of these annotated references will become available for .NET Framework classes outside of the BCL.
![]()
Recently, I have been involved in a number of discussions on Transactions, Transaction management, the different approaches to how Transactions can be performed, as well as the different types of Transactions.
It a very interesting area, as it is a part of computer science that has become such an integral part of Software development today. Sometimes when we think of Transactions, we think about ACID Transactions or perhaps associate Transactions as being a software development tool/pattern/service that only applies to Databases. While both are somewhat correct, they don't really explore the feature richness of Transactions or how they can be applied to so many scenarios outside of those that involve the transformation of state where databases are involved.
When I think of Transactions, I like to think of them as being a collection of patterns, interfaces and contracts that help us ensure the correctness of data/state, as we transform from one known state to another. For me, it is this collection of patterns, interfaces and contracts that make up the transaction paradigm and help us as developers and architects tackle one of the most difficult areas in software development, i.e. Concurrency management and Recovery control.
The reason that I like to think of Transactions this way, is due to the way that some commonly held views on Transactions have developed over time.
Some of these include:
- Transactions only apply to databases
- The many variances of the definition of the ACID properties, especially that of "D". (I'll expand on this in a later post)
- how the "commit" or "rollback" aspects of the paradigm can be sometimes viewed. Does the "Rollback" always have to be the inverse of every action that makes up the unit of work?
- Volatile Transactions can be seen as not being transactional. Same train of thought as that on some of the definitions of the ACID properties of Transactions that exist. (I'll expand on this in a later post)
Just me thinking out loud really... but I welcome any comments!
![]()
|