Today, as we write application code
we generally are not dealing with immutable state/data, we are applying application logic to state/data so that it transforms from one known state to another known state in a consistent, all or nothing fashion. Today, when performing this in code that does not involve Transaction Managers or Resource Managers, we use Try/Catch exception handling statements to mimick the "Commit" and "Rollback" concepts of transactions. The difficulties with such an approach is that as we scope certain types of work and/or scope more than a few simple actions into a unit of work, the level of complexity in nested try/catch code grows and can easy reach a point where it cannot truely provide the "Commit" and "Rollback" concepts of a Transaction.
For example,
take the Try/Catch below
try
{
// operation(stateX)
// operation(stateY)
// operation(stateZ)
}
catch
{
// cleanup after states X, Y, and/or Z, as something has gone wrong and we need to get to a previously known state
}
The challenges with using try/catch in a Commit/Rollback style are some of the following:
-
If stateX is shared between between more that one execution contexts, another execution context may read an inconsistent state and hence a "dirty" read, as the scoped unit of work may fail
- What if an operation on stateY deletes/alters a resource and how do we "Rollback" in the catch.
- How do we coordinate with resources that are transactional, i.e. if stateX represents a call to a transaction queuing/messaging or database system. How we coodinate between the different resources and unit of works!
- and there is lots of other problems
While all of the above can be eventually handled in code, with careful planning and structure, but what is the essence of what we are trying to achieve in the code and does it form a pattern! It is the "Commit" and "Rollback" concepts of a Transaction and if we factor the patterns, interfaces, contracts and interactions of Transacional Processing into the code we have more flexibility and agility. Please note I am not suggesting that every try/catch has a "Commit" or "Rollback" focus, but we sometimes use try/catch to group actions into a unit of work in which we determine the outcome of a unit of work by interrogation (using if statements) and do not negotiate/coordinate the outcome of the unit of work (as an example). Also, to provide the most flexibility and agility, we need for the actions that are grouped into a unit of work to be able to be aware of the (transactional) context. We also have the problem that try/catch is not context aware, i.e. it only knows a single execution context, and while dealing with a single execution context may suit us some times, we need to be able to handle both concurrency management and recovery control at the same time!
One approach to providing a cleaner implementation, is to factor out the interrogation and make it the negotiation/coordination of the outcome from the application logic and for an intermediary to work as a coordinator between the resources being used and the application logic. The coordinator controls the enlistment of resources within a scoped execution context (as defined by the application code) that we want to be transactional. We then need to define a number of common interfaces and interaction semantics, so that the use/enlistment of one or more resources in the scope/context of a unit of work is clearly defined. This can be achieved by having methods/semantics/behaviors common to all resources so that the different stages of moving/transforming from one known state to another known state as a unit of work or as a part of a unit of work, happens in an all or nothing fashion.
All of this fits quite nicely with the 2 phase commit (and even the one phase commit) protocol as to how the "Commit" or "Rollback" semantics of units of work can be achieved.
Oppps ....... Hmmmm ..... since I thought I was going to be able to do this is 3 to 4 postings, I seem to have underestimated and need one or two more. I think I will leave it here for now with this posting and follow up with a part 4,5,6
![]()
When thinking about Transactions, I think it is fair to say that Transactions are one of the most powerful and frequently used paradigms in Enterprise Computing today. With Transactions, we have a collection of patterns that deal with how concurrency management and recovery control can be handled in possible shared/concurrent environments/contexts and it is this that ensures data/state correctness. It is the key concepts and abstractions of Transactions maintaining data/state consistency in possible concurrent state/data access environments (where failures may occur) that have allowed us as developers and architects to be largely released from having to deal with the tricky aspect of maintaining the correctness of state in concurrent environments in an enterprise application.
Two of the very interesting aspects of Transactions is how trusted they have become and how we perceive them. Concurrency Management and recovery control are some of the more tricky aspects of Enterprise Computing, but yet when leveraging the Transactional paradigm and executing units of work in a concurrent environment in the context of a transaction, we are not overly concerned about isolation or failure as much of the work is done for us by the Transaction Paradigm. This is a testament to the level of trust and adoption that Transactions have achieved as an abstraction and how it has evolved into a framework of runtime services that takes most of the responsibility for concurrency management and recovery control from us as developers and architects. The paradigm of Transactions can be seen as a set of patterns, Interfaces and Contracts that can be leveraged to simplify the transformation from one known state to another and without these Interfaces and Contracts; a lot of code, time and effort has to be spend in making sure that data/state transformations happen in a consistent and all or nothing fashion.
It is these key abstractions in the form of runtime services or frameworks that have provided great productive gains for us as developers and architects, as it allows us to focus on business logic and allows for the underlying runtime services to deal with concurrency management and recovery control. Therefore as we implement application logic, by applying units of work to state, we want the transformation from one state to another to be applied in a consistent (all or nothing) fashion, as well as wanting to be isolated from other parallel activities. As I mentioned in my
How should we think of Transactions ?
blog posting, I like to think of the Transaction paradigm as a collection of patterns, interfaces and contracts that can be leveraged to tackle the issue of data/state correctness as we group actions into unit of works to transform from one known state to another. Some of this collection of patterns, interfaces and contracts are short running transactions that in the main adhere to the ACID properties but there is a rich portfolio of transactional types that we do not focus on that much and/or maybe perceive as not meeting the ACID properties (see
How should we think of Transactions ?
and
Volatile Transactions & In-Memory Transactions - do they have ACID, ACId or ACI properties?
)
Today, the most common example of how the Transaction paradigm can provide concurrency management and recovery control is Database Management Systems (DBMS). Databases (aka DBMS) are themselves another key and often used architectural piece of Enterprise computing. Database Management Systems (DBMS) provide a paramount role in how an Enterprise may store, retrieve and share state/data. But we sometimes draw an association between Databases and Transactions that is so close that they are often perceived as being interchangeable, i.e. Transactions are DB Transactions and nothing else. The close associations between databases systems and transactional processing have been fundamental in bringing the concept of Transactional Processing to the fore, but in my honest opinion I think that the close association may have also brought a narrowing of perception of what are Transactions and Transactional Processing.
I have put together a series of blog postings to share my current views/thoughts! After talking to a few folks about Transactions, it is very interesting to hear the different views and I'm not sure if there is a one size fits all for what is a transaction, but this is the thing about a paradigm and a snappy acronym, it can have a life of its own
Maybe we interchange the paradigm, acronym’s and implementations into one fixed view and its this fixed view that maybe causing the issue!
![]()
In my last blog posting,
How should we think of Transactions ?
I mentioned Volatile transactions and how I currently think they can be perceived in different ways! Do they have ACID, ACId or ACI properties?
In case you do not know what Volatile Transactions (aka In-Memory Transactions) are, they are transactions that do not persist to a durable store such as a hard disk (ouch! we have multiple definitions of durable to content with
). They are transactions that are done completely in memory and do not incur the I/O cost of writing to a hard disk a Transaction log or the actual known data/state.
What is interesting about Volatile transactions is that they are sometimes perceived as not being Transactional, because they are viewed as not meeting the ACID properties, especially the "D" property. I suspect the main reason for this view is because there are no Transaction logs and/or no data stored to hard disk.
So when you look at a typical definition of the ACID properties (like what I have listed below), some of the reasons why this view maybe held become apparent:
(NB. this is an amalgamation/synopsis of many definitions I have seen)
A
- Atomicity - Transactions are atomic (all or nothing)
C
- Consistency - The transaction is a correct transformation of state. Consistency constraints that are defined on the underlying data servers are preserved by a transaction.
I
- Isolation - Each transaction should appear to execute independently of other transactions that may be executing concurrently in the same environment.
D
- Durability - Once a transaction commits, it’s updates survive even, if the system goes down
ok, so a Volatile Transaction meets the ACI of the ACID properties, but not the D as defined above. But is this ("Once a transaction commits, it’s updates survive even, if the system goes down") a true reflection of what Durability means? Many similar variants of this definition gets used on a frequent based and they bring specifics to the definiton that are not about the failure of meeting the ACI properties or making the "Commited" value available/public but focus on the failure of media or a system. In my opinion the failure of media or a system is not an aspect of the paradigm or indeed part of the ACID properties, but an aspect of how the paradigm can be implemented and/or a requirement of the longevity of the state being persisted/determined. It is perhaps the fact that the best known implementation of the data server feature of the transactional paradigm is a database that has led us to shape the ACID properties to the specifics of databases. But perhaps the Transactional paradigm has more to offer than the correctness of data/state transformations on a database. The transaction paradigm has a core focus on simpilifing how we can achieve the correctness of data/state transformations where we may have to deal with concurrency management and recovery control and this can be applied to any transformation of state.
Perhaps a different way to look at the durability of ACID is where the definition of ACID as an acronym came from and how some of the leading experts in the industry define them. As far as I know the ACID acronym was first coined by Haerder and Reuter (1983). The definition I like to use is that by Gray and Reuter (1993). It is the following:
A
- Atomicity - A transaction's changes to state are atomic: either all happens or none happen. These changes include database changes, messages, and all actions on transducers.
C
- Consistency - A transaction is a correct transformation of the state. The actions taken as a group do not violate any of the integrity constraints associated with the state. This requires that the transaction be a correct program.
I
- Isolation - Even though transactions execute concurrently, it appears to each transaction, T, that others executed either before T or after T, but not both
D
- Durability - Once a transaction completes successfully (commits), its changes to the state survive failures.
When you read this definition of Durability (and the ACID properties overall), it in my option explains the feature richness of transactions and how it can be applied to the persistence of the transformation of state and also how it applies to the other properties such as A, C and I. The state survives failures by the data server ensuring that when the transaction commits a new durable state is persisted/stable/available/public and if something goes wrong (such as not be able to ensure Consistency) that the data server returns to the previous known state. In essence, this is a contract that guarantees the persistence and stability of the state and that the outcome of a transaction is known/public and usable in subsequent transactions, as a known/correct state. A good definition I have seen is that once a transaction is committed, it cannot be abrogated.
With Durability being about the persistence, stability and making public of “commited”/known state as we transform from one known/correct state to another, it allows us to leverage the transaction paradigm in many scenarios. For example when using Volatile Transactions, we are leveraging data servers that store data on Volatile media, such as RAM and providing the transaction paradigm in-memory. With the common misunderstanding that Transactions are exclusively for databases, the question arises about the “D” property.
For me the Durability aspect of a transaction is present in a Volatile transaction. The question is whether its “D” or “d”. Since most transactions are based on the X/Open Transaction processing model, the requirement for log managers or lock managers in resource managers (data servers) are not central or core to implementing the transactional paradigm at a base level. In relation to the usage of Volatile media for Transactions? in my opinion the requirement of the media used by the data server is that it is able to present a stable/persistent/public state once "Commit" has occured.
So I believe that Volatile Transactions meet all of the ACID properties and that its ACID with a "D"
phew! long post.
I guess the crux of the issue is the 2 uses of "durable" when talking about Transactions. One is the "D" in the ACID properties and the other is the property of the data/state/media being used (and how it is stored). The "D" in the ACID properties is more about making a "commited" value available and the other is about an implementation requirement of how long (and/or resilient) a known/committed state must be.
![]()
The stack trace below (snapshot taken at a breakpoint in [WebMethod] "HelloWorld") shows that I am having quite a bit of programming fun these days. Server-side ASP.NET hooked up to a MSMQ listener.
simpleservicerequestinweb.dll!SimpleServiceRequestInWeb.Hello.HelloWorld() Line 53 C#
system.web.services.dll!System.Web.Services.Protocols.LogicalMethodInfo.Invoke(System.Object target, System.Object[] values) + 0x92 bytes
system.web.services.dll!System.Web.Services.Protocols.WebServiceHandler.Invoke() + 0x9e bytes
system.web.services.dll!System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest() + 0x142 bytes
system.web.services.dll!System.Web.Services.Protocols.SyncSessionlessHandler.ProcessRequest(System.Web.HttpContext context) + 0x6 bytes
system.web.dll!CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionStep.Execute() + 0xb4 bytes
system.web.dll!System.Web.HttpApplication.ExecuteStep(System.Web.HttpApplication.IExecutionStep step, bool completedSynchronously) + 0x58 bytes
system.web.dll!System.Web.HttpApplication.ResumeSteps(System.Exception error) + 0xfa bytes
system.web.dll!System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext context, System.AsyncCallback cb, System.Object extraData) + 0xe3 bytes
system.web.dll!System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest wr) + 0x1e7 bytes
system.web.dll!System.Web.HttpRuntime.ProcessRequest(System.Web.HttpWorkerRequest wr) + 0xb0 bytes
newtelligence.enterprisetools.dll!newtelligence.EnterpriseTools.Msmq.MessageQueueAsmxDispatcher.MessageReceived(System.Object sender = {newtelligence.EnterpriseTools.Msmq.MessageQueueListener}, newtelligence.EnterpriseTools.Msmq.MessageReceivedEventArgs ea = {newtelligence.EnterpriseTools.Msmq.MessageReceivedEventArgs}) Line 33 C#
newtelligence.enterprisetools.dll!newtelligence.EnterpriseTools.Msmq.MessageQueueListener.ReceiveLoop() Line 305 + 0x2b bytes C#
>
(c)2002-2004 Clemens Vasters.
You want to build service oriented applications on the Microsoft .NET Frameworkand are not quite sure how?
newtelligenceWorkshops
get you going with concrete, pragmatic guidance on today's shippingtechnology stack, keeping tomorrow in perspective. Available on-site, world-wide.
![]()
The little series I am currently
writing here
on my blog has inspired me to write way too more code than actually necessary to get my point across ;-) So by now I've got my own MSMQ transport for WSE 2.0 (yes, I know that others have written that already, but I am shooting for a "enterprise strength" implementation), a WebRequest/WebResponse pair to smuggle under arbitrary ASMX proxies and I am more than halfway done with a server-side host for MSMQ-to-ASMX (spelled out: ASP.NET Web Services).
What bugs me is that WSE 2.0's messaging model is "asynchronous only" and that it
always
performs a push/pull translation and that there is no way to push a message through to a service on the receiving thread. Whenever I grab a message from the queue and put it into my SoapTransport's "Dispatch()" method, the message gets queued up in an in-memory queue and that is then, on a concurrent thread, pulled (OnReceiveComplete) by the SoapReceivers collection and submitted into ProcessMessage() of the SoapReceiver (like any SoapService derived implementation) matching the target endpoint. So while I can dequeue from MSMQ within a transaction scope (ServiceDomain), that transaction scope doesn't make it across onto the thread that will actually execute the action inside the SoapReceiver/SoapService.
So now I am sitting here, contemplating and trying to figure out a workaround that doesn't require me to rewrite a big chunk of WSE 2.0 (which I am totally not shy of if that is what it takes). Transaction marshaling, thread synchronization, ah, I love puzzles. Once I am know how to solve this and have made the adjustments, I'll post the queue listener I promised to wrap up the series. The other code I've written in the process will likely surface in some other way.
(c)2002-2004 Clemens Vasters.
You want to build service oriented applications on the Microsoft .NET Frameworkand are not quite sure how?
newtelligenceWorkshops
get you going with concrete, pragmatic guidance on today's shippingtechnology stack, keeping tomorrow in perspective. Available on-site, world-wide.
![]()
"My Lists", "My Photos", "My Profile" .... sounds all very familiar over there in
MSN Spaces
. So ... roll in the Web service interfaces, please.
(c)2002-2004 Clemens Vasters.
You want to build service oriented applications on the Microsoft .NET Frameworkand are not quite sure how?
newtelligenceWorkshops
get you going with concrete, pragmatic guidance on today's shippingtechnology stack, keeping tomorrow in perspective. Available on-site, world-wide.
![]()
The Hands On Lab materials for WSE have been updated for the WSE 2.0 SP2 release and now include VB.NET code as well as C#. If you're looking to get started with WSE 2.0, these provide some great grounding material....
![]()
Service Pack 2 is final and available on MSDN. There was one additional fix between the Pre-Release and the final build involving policy processing for clients that do not send the WS-Addressing headers. In this case, request policy was being applied correctly but response policy would not be applied. Please report any issues you find on the product to wsefeed....
![]()
It is decided
: we will release
WSCF
0.4 with a new command line interface to all code generation features - we tested this command line tool in a few real world projects and it seems stable enough to be distributed ;)
Version 0.4 also comes with a
WSDL Wizard
to design and create your Web service's message and interface contract.
Of course, the tool will remain free. It is our community contribution. Thanks.
![]()
With over 40 Webcasts on: why it matters to the business, common strategies and methods, and guidance on specific implementation scenarios between the major platform players, this is going to be quite a comprehensive series.
You can pre-register for the Webcast series at
http://www.interopmonth.com/
and in a fews weeks you should receive a schedule on the sessions, speakers and topics.
Some highlights you can expect:
·
David Chapell (.NET)
and
David Chapell (J2EE)
to key note the webcast series and end the series.
·
Information on managing a multi-platform data center
·
Information on integrating Java with Windows applications and clients
·
Using web services to extent the life of legacy systems and provide efficient information exchange and functionality
![]()
If you have a look at the
Transaction Specification Index Page
on MSDN, you will see that:
- WS-Coordination was updated in November 2004.
- WS-AtomicTransaction was updated in November 2004.
- WS-BusinessActivity was updated in November 2004.
They are well worth a read.
Also, Luis Felipe Cabrera, Christopher Kurt and Don Box have updated
"An Introduction to the Web Services Architecture and Its Specifications"
![]()
Just spotted a reference to Dublin on the ASP.NET feed, so I had to have a look!
Excellent!!
Christian Weyer
is coming to town and talking at a
IrishDev user Group
event and is covering a really good topic,
"Visual Studio .NET's Web services wizardry reconsidered: implementing your Web services the other (the right?) way"
.
This is really worth attending, as I have always found Christain's sessions to be very good and informative, especially his talks about Web Services and WSDL design.
Christian is a very strong proponent of the WSDL first and or Schema first approach to the developement of Web Services. WSDL/Schema first based development is something I 1st came across a few years ago and it was more out of necessatity (as I wanted to reuse schema) than design that led me down this road, but since then I have never looked back.
If you are interested in getting some more details on WSDL/Schema first based development, try some of the following links, I think they are very useful:
and of course,
Christain's blog
is a good place as well!
![]()
|