Friday, 29 November 2013

Using Flyway with OSGi (Part One)

I spent a lot of time making flyway work in an OSGi container (felix) and I'd like to share my findings. Felix support in flyway was added not too long ago, and there is not much information to be found on the website (yet).

My main sources were this issue, this github example project and, of course, the flyway source code.

P.S. I hope you like lengthy blog posts ;) And, sorry, no pictures.


[where I tell you what I want to achieve]

What I want to create here (in the first part about flyway with OSGi) as an example is a bit more complicated than the before-mentioned github example project:

I want to have 
  • a bundle with an entity, a persistence unit and a repository
  • a couple of SQL scripts defining the database and initial data
  • an integration test case actually calling the repository. 
(Part two will add another bundle with another entity which has a manyToOne relation to the first one).

The example

[where you will find the code]

If you want to check out the glory details, you can find the code for the example here. If you use eclipse with bndtools, please create a new workspace, as a cnf folder is part of the example (and would clash with a second one).

The projects you care about are the ones in the osgi/flyway/contacts subfolder.

Import them into your IDE, and, assuming you are using bndtools, everything should be fine. (Otherwise, run "ant build" in both examples.osgi.flyway.contacts.* subdirectories).

First test

[where you can make sure it does what it should]

In eclipse, with bndtools, you can right-click on the project ("it" stands for integration test) and run it as "Bnd OSGi Test Launcher (JUnit)". On the console, you should see that the test has passed, and, after refreshing the project, you should see a new folder "exampleContactsDb" together with some new log and DDL files (ending with .jdbc).

The same can be achieved by running "ant test" in the same directory.

The example uses an embedded derby database, and you can use the usual tools to look inside the databases contents if you wish.

Some background and some impediments

[where I provide some information why I did it that way]

In order to make the example work, I had to do some investigation first (skip this section if you only want to see how it is done in code. But I do recommend reading this nevertheless ;)).

It seems like the only way to utilize flyway in OSGi is by using fragments (sigh!). So, the contacts bundle, containing the DDL and SQL scripts should be a fragment attached to the com.googlecode.flyway.core bundle. On the other hand, I need to execute some code when the bundle starts in order to initialize the flyway update or init process. For this, I need to create (or reuse) a datasource.

For proper reuse, the datasource (or the initialization properties for a new datasource) should not be defined inside the bundle, but be provided from the outside, of course (e.g. using some kind of "datasourceProvider" or by passing the properties from the config admin implementation). So, being inside an OSGi container, I should use some OSGi service to provide the needed data.
But, as the OSGi compendium states in 112.4.2 ("Service Component Header")
"A Service-Component manifest header specified in a fragment is ignored by SCR"
So, using declarative services (aka SCR) and defining a fragment will not work for me here. [A resort could be to use a blueprint service (see this stackoverflow question) - but I want to stick to SCR and not introduce another technology to manage my services (and end up with both felix SCR and some blueprint bundles)].


[where I tell you how I overcame the obstacles]

So, what to do? I need a fragment bundle and use services. Well, let's have two bundles then. The first one (contacts.core) will contain the flyway initializer, the repository to access the database and the persistence unit definition. The second one (contacts.db) will define the DDL and initial SQL scripts.

Fortunately, using bndtools, I will not have to define two projects for the two bundles: one project with two bundle descriptors will do the trick (remember, the second bundle in the example is the integration test!).

The glory details

[where you find what you need to know to do it yourself]

In this section, I will show or reference all the important bits of the example project and give you some information about why I did things the way they are. (Btw, I don't mind getting feedback telling me that things could be done in a different, more elegant way ;))

All the files mentioned here are in the examples.osgi.flyway.contacts project.

1. The persistence unit

The usual persistence.xml file is defined in the resources/META-INF folder and will be copied (by bnd) into the contacts.core bundle. There is nothing (really) special about that file, it defines the "ContactsDB" persistence unit with transaction-type Resource-Local (as recommended for OSGi JPA). It lists the entities and takes care about the database connection properties.

2. The bnd file

This is the entry point for the bnd library.

In -buildpath, I define all the bundles necessary for compilation of the contacts project (the referenced jars are provided in the cnf project). 

In -sub, I let bnd know that I want to create more the one bundle from this project: Each other *.bnd file found in the project will define the contents of a derived bundle.

3. The core.bnd file

This file defines the contents of the examples.osgi.flyway.contacts.core bundle.

Include-Resource adds the resources directory to the created bundle, and the other instructions will make sure our MANIFEST.MF file will look the way it should. Nothing else will end up in the contacts.core bundle.

Service-Component: * will instruct bnd to scan the exported packages for @Component annotations. The annotated classes will be used to define OSGi declarative services automatically. They will be created during the build and end up in the OSGI-INF directory of the generated bundle.

Important: Do not forget to add the org.osgi.framework Import-Package instruction. Without, flyway will give you a "unsupported protocol: bundle" error).

4. The db.bnd file

Similar to the core.bnd file, this file defines the contents of the examples.osgi.flyway.contacts.db bundle.

Again, some (but not all) resources of the project are copied into the generated bundle, and the fragment host is defined

5. The FlywayInit class

There is a little trick involved making flyway actually do what I want it to do (as of version 2.2.1):
Replacing the ContextClassLoader before giving control to flyway (and then, after that, reverting that change again) helps flyway finding the files it searches for.


[where I..., well, you guessed it]

As I said, I spend quite a lot (too much?) time getting this to work, and I still have Part II ahead of me (which adds some more complexity and requires a workaround which I am not really happy with).

As I didn't find any tutorials going into this depth, I had to write one myself, and I guess, if you are still with me here, you liked it. At least I hope so. Give me your feedback or your insights, and spread the word.

See you in Part II.