Functional Transaction Management – old dog, new tricks! 3 comments


This blog post is all about the new Transaction Control service which is proposed to be part of the next release of the OSGi Enterprise Specification. Paremus has been leading this specification work, which arose from a collaboration with one of our customers, McCarthys. The current state of the RFC is available on GitHub, and there’s even an implementation available in Apache Aries.

Before we delve into the cool features of the Transaction Control service, it’s probably worth remembering why we need yet another transaction abstraction…

A Short History of Transaction Management

Software controlled transactions have existed for a long time — commercial products that are still available now can trace their origins back to the 1960s. Since that time a lot has changed, first we saw the rise of C, then of Object Oriented programming, then of the Web, and now of Microservices.

Over the same time period there was one significant change to the way that transactions were managed – originally transaction boundaries had to be explicitly declared.

    transactionManager.begin()
    try {
        // Work goes in here
    } finally {
        transactionManager.commit();
    }

Unfortunately it turns out that properly handling these transactions is complex, for example the snippet above is not sufficient to rollback if the work throws an exception!

In addition to the complexity of managing transactions correctly, imperative transaction management also adds a lot of noise to the application code. A three line business method can rapidly grow to fill your screen if you have to suspend an existing transaction, start a new transaction, complete the transaction, and resume the original transaction.

 

Declarative Transaction Management

Recognising that imperative transaction management was a nightmare for developers, the creators of Application Servers (and later the Java EE specifications) designed declarative ways to manage transactions. This avoided the user needing to write any code, dramatically simplifying the model.

    @TransactionAttribute(REQUIRES_NEW)
    public void doWork() {
        // Work goes in here
    }

For many years this model was seen as the gold standard for transaction management. It eliminated mistakes, and it simplified code. There were, however, some problems…

Container Dependencies

The original EJB transaction management required specific interfaces to be implemented, specific classes to be extended, and pre-deployment steps before code could run, each of these things adds its own separate layers of complexity to the code, and ties your application to a container.

The rise of the Spring framework was a reaction to this complexity, and the heavy-touch management of Java EE. Instead Spring focussed on “pure POJO” programming, designed to make your code easily portable, runnable and testable inside or outside the container.

While Spring did a much better job of hiding the dependencies, the fundamental problem with a pure declarative approach is that there must be a container somewhere. Without a container there is no code to start or end the transaction. Spring’s promise of simpler, container independent components was only possible if you were still implicitly tied to the Spring container, even when running tests. With Spring’s transaction management we’ve just swapped one container for another.

Fun with Proxies

Another problem idiosyncrasy with declarative transaction management is that it must proxy your POJO in order to work, and that means that not everything will work as you expect. Firstly, some proxying is limited to interfaces only, meaning that you can only have transaction management on interface methods. In other cases the container may be able to use subclassing, which allows transaction management on public and protected methods, but still not on private or final methods!

You may be able to live with these limitations, however there is a much more insidious problem with declarative qualities of service added through proxying.

The proxy must be invoked to trigger the quality of service

What does this actually mean? To put it simply, if you call my object’s foo() method from outside the proxy it will start a transaction. If I call my object’s foo() method from inside the object then it will not start a transaction!

    @TransactionAttribute(REQUIRES_NEW)
    public void doWork() {
        // Work goes in here
    }

    @TransactionAttribute(SUPPORTS)
    public void doWork() {
        // A bit of Non transactional work goes in here

        // This internal call does not hit the proxy, 
        // so no new transaction is started
        doWork();
    }

Enlisting Resources in the Transaction

The third big problem that people encounter with declarative transaction management is that it becomes incredibly hard to work out whether the resource that you’re using is actually participating in the transaction or not. If I get a connection from a DataSource then it is the container, not my application, that makes sure the connection is linked to the transaction. While it is good to avoid adding this code explicitly, it does mean that my application is reliant on yet more “container magic” behind the scenes. If I use a non-JTA datasource, or I forget to attach my datasource to the Spring PlatformTransactionManager, then my resource access will not be transactional!

Fixing the Problems with Declarative Transaction Management

The big problem with declarative transaction management was that it tried to take away too much from the application code, replacing it with “container magic”. The problem with relying on magic is that the resulting system ends up being more complex, not less. We therefore should be aiming to simplify and minimise transaction management code, not eliminate it entirely.

 

Explicit Transaction Management Using Scoped Work

With the advent of Java 8 lambda expressions and functional interfaces, we now have a new, better way to define transactional work. Rather than a messy try/catch/finally block, we can now simply pass a closure to be run in a transaction!

    txControl.required(() -> {
            // Work goes in here
        });

As the work we pass is actually a closure (not just a function) it can capture variables and state from its context. For example this closure captures the parameters passed to the saveMessage method.

    public void saveMessage(String user, String message) {
        txControl.required(() -> {
                PreparedStatement ps = connection.prepareStatement(
                        "Insert into MESSAGES values ( ?, ? )");
                ps.setString(1, user);
                ps.setString(2, message);
                return ps.executeUpdate();
            });
    }

 

Explicit Resource Enlistment

While explicitly enlisting resources with every transaction is cumbersome, there is value in explicitly linking a resource with the transactions runtime. This link means that the resource can be relied upon without any magic in the background, whether running in a unit test, or having been migrated to a pure Java SE microservice. The Transaction Control Service creates this link using a ResourceProvider. A ResourceProvider is a generic interface that is usually specialised to return a particular resource type. For example the JDBCConnectionProvider provides a javax.sql.Connection resource object when connected to a TransactionControl service.

The important thing about the resource objects returned by the provider is that they are thread-safe, and automatically integrate with the scope of the current thread. In particular, you don’t need to worry about tidying up the resource. It can be retrieved once and then cached and used in any piece of scoped work. For example the following Declarative Services component provides a transactional service using JDBC and the Transaction Control service.

    @Component
    public class MyDaoImpl implements MyDao {

        @Reference
        TransactionControl control;

        Connection dbConn;

        @Reference
        void setResource(JDBCConnectionProvder provider) {
            dbConn = provider.getResource(control);
        }

        @Override
        public void saveMessage(String user, String message) {
            control.required(() -> {
                    PreparedStatement ps = dbConn.prepareStatement(
                            "Insert into MESSAGES values ( ?, ? )");

                    ps.setString(1, user);
                    ps.setString(2, message);

                    return ps.executeUpdate();
                });
        }

        @Override
        public void getMessagesForUser(String user) {

            return control.supports(() -> {
                    PreparedStatement ps = dbConn.prepareStatement(
                            "Select MESSAGE FROM MESSAGES WHERE USER = ?");

                    ps.setString(1, user);    

                    List<String> result = new ArrayList<>();

                    ResultSet rs = ps.executeQuery();

                    while(rs.next()) {
                        result.add(rs.getString(1));
                    }

                    return result;
                });
        }
    }

 

Simpler Resource Lifecycles

You may have noticed that the previous example doesn’t close any of the JDBC resources. Normally this would trigger all sorts of unpleasant leaks, however in this case we actually can rely on the Transaction Control service. Because we have explicitly defined a scope using a closure, it is clear when that scope ends. At the end of the scope all of the resources used in that scope are notified, and can tidy themselves up. In many ways this is like a try-with-resources block:

    try(Connection c = getConnection()) {
        // Work goes in here
    }

The main advantage of scoped work over try-with-resources is that we don’t need to know which resources are going to be used in the scope. This is particularly useful when calling out to external services.

In Summary

The Transaction Control Service is a new way of thinking about Transaction management. By applying functional techniques we can enable simple, reliable transaction management without the need for any containers or proxying. This shift enables microservices to be written more maintainably, and with fewer dependencies.

For more information please do look at the Apache Aries site, and at the OSGi RFC. Real implementations for transactional JDBC and JPA are available now — feel free to try them out!

Share This:
Facebooktwitterredditpinterestlinkedinmail

Leave a comment

Your email address will not be published. Required fields are marked *

3 thoughts on “Functional Transaction Management – old dog, new tricks!

  • Balazs Zsoldos

    I agree with the statements that annotations and proxies are not good to solve transaction handling. I also agree with the statement that the code will be much cleaner if we use lambda expressions to define lambda expressions to specify transaction propagation within our functions.

    However, I do not think it is worthy to avoid JTA. In the last decades, everyone had to implement XAResource to make a technology transaction aware. There are lots of solutions that support it: JDBC DataSources, Queues, Maps, etc. If you specify a new interface that does the same, you will find yourself fighting a much bigger battle than you should. You will have to support each technology, that supports JTA by default.

    If you get the database connection outside of your transactional lambda expression, I do not see how you will implement “requiresNew”. You could only do that if the database supports XA, but even in that case, it will not be easy to implement it without JTA. Here is how it works with commons-dbcp: You must always get the connection within the transaction scope. If a requiresNew is called, the pool will give you a new connection until the end of the new scope. The connection does not become suddenly transactional if someone starts a transaction after getting the connection.

    What I did years ago is that I implemented a simple TransactionPropagator class that accepts functional interfaces. We use lambda expressions with JTA since Java 8 is out, and we have never felt the necessity of additional interfaces. Enlisting the XAResource is done by the technology itself or by a wrapper like commons-dbcp BasicManagedDataSource. If you are interested, the latest versions of the sources are here: https://github.com/everit-org/?utf8=✓&query=transaction-propagator

  • Tim Ward Post author

    Hi Balazs – thanks for reading the post!

    > However, I do not think it is worthy to avoid JTA. In the last decades, everyone had to implement XAResource to make a technology transaction aware. There are lots of solutions that support it: JDBC DataSources, Queues, Maps, etc. If you specify a new interface that does the same, you will find yourself fighting a much bigger battle than you should. You will have to support each technology, that supports JTA by default.

    I fully agree with this statement, which is why the TransactionControl service uses XAResource! If you look at the RFC document, or at the API https://github.com/apache/aries/blob/trunk/tx-control/tx-control-api/src/main/java/org/osgi/service/transaction/control/TransactionContext.java#L116 then you can see that it is perfectly possible to register XAResource implementations with the ongoing transaction. This is actually what the Aries implementation does to support XA enabled JDBC.

    > If you get the database connection outside of your transactional lambda expression, I do not see how you will implement “requiresNew”. You could only do that if the database supports XA, but even in that case, it will not be easy to implement it without JTA. Here is how it works with commons-dbcp: You must always get the connection within the transaction scope. If a requiresNew is called, the pool will give you a new connection until the end of the new scope. The connection does not become suddenly transactional if someone starts a transaction after getting the connection.

    I agree that the connection must always be associated with the transaction, however in this case you seem to be assuming that the connection is retrieved in the call to getResource(). This is actually not the case. From the blog post:

    “The important thing about the resource objects returned by the provider is that they are thread-safe, and automatically integrate with the scope of the current thread. In particular, you don’t need to worry about tidying up the resource.”

    The connection resource that you get back is not mapped 1:1 to a physical connection. The underlying connection used depends on the Transaction Context. Implementing requiresNew is therefore easy, as you have a brand new connection to work with. This model therefore works without JTA as well as with it.

    > What I did years ago is that I implemented a simple TransactionPropagator class that accepts functional interfaces. We use lambda expressions with JTA since Java 8 is out, and we have never felt the necessity of additional interfaces. Enlisting the XAResource is done by the technology itself or by a wrapper like commons-dbcp BasicManagedDataSource.

    This project was one of the sets of ideas that fed into the OSGi RFC. One of the primary drivers was to allow lambdas, but we also wanted to manage the resource lifecycle (no try/finally/close), to support non XA usage and to have a richer API for rollback. This added up to a thin, but necessary, API.

  • Balazs Zsoldos

    Hi Tim,

    thanks for the answers!

    > I agree that the connection must always be associated with the transaction, however in this case you seem to be assuming that the connection is retrieved in the call to getResource(). This is actually not the case…

    I see an issue here, but it can happen that I misunderstand something:

    All code checkers like Findbugs or PMD will warn you that you should call close on every instances that implement the Closeable interface. By saving the try-with-resources statement (two lines), code checkers and other developers cannot be sure that they see the right behavior or a bug just by looking at the code.

    > but we also wanted to manage the resource lifecycle (no try/finally/close), to support non XA usage and to have a richer API for rollback. This added up to a thin, but necessary, API.

    If someone wants to integrate with the JTA lifecycle, it is enough to implement XAResource interface and the enlisting logic. The technology itself does not have to be XA capable if there are no more transactional sources or high do not need high consistency. I have never understood why there is a different Transaction API in Spring to achieve the same functionality as JTA already had. The API of JTA is a bit ugly, but one has to touch it very rarely (or never). Here is a simple example, how a transactional Map is wrapped to become Managed by JTA: https://github.com/everit-org/transaction-map-managed/blob/master/src/main/java/org/everit/transaction/map/managed/ManagedMap.java. I did not need to specify a new API to make the Map managed. The solution will work in every JEE containers as well as in OSGi if there is a TransactionManager.