Porting DBLX to the Android platform turned out to be the most simple part of the exercise. It took minutes to port DBLX for use on Android, but then we had to spend weeks working out the packaging writing the UI components.
If you plan to port an application to Android, or create an Android application you may find some of the following WTF's very useful.
How will you know if your Java application can be ported to Andriod?
If your Java application uses JDK 1.6 and has few external dependencies, it will likely port to Android with little to no code changes required. The way you determine if your app will work is to create a new Andriod project in Eclipse and then import the source code for your application. Once the import is complete then Eclipse will attempt to compile everything, and if there is a compatability issue you will be notified immediately. If your Java app has dependencies on JAR files, then it is likely that you will have to port more than just your application to Android.
My Java app uses apis from JDK 1.7. Will it port to Andriod?
From what we can tell, you are fucked. Not only do JDK 1.7 apis not appear to be compatible with Andriod, we have had significant problems with JDK 1.7 in general. DBLX runs with Java apis which were in Java 1.6 and earlier. Java 1.6 seems to be where your application needs to be if you want to port to Andriod.
Is there anything you can do with your Java code to ensure it works seamlessly with Android?
We learned a few things about Java and Android that made the porting process extremely simple:
- Use pure Java Buffered IO streams: If you are using java.nio or using an external IO library you may not find an equivalent in the Android apis.
- Use org.w3c package references for all XML manipulation: When you use org.w3c packages for your XML parsing then Java uses the compatible XML library found in the classpath. So if you want to use Xerces as your XML parser you would use org.w3c package names in your code and make sure all the Xerces jar files are in your classpath.
By using the org.w3c packages for XML parsing you will get the same behavior between Java and Andriod and not have to change one line of code. Since DBLX is based on XML, this was a huge win for us.
- Use JDK 1.6 or earlier: Your app really needs to be completely Java 1.6 compatible to assume you won't have to rewrite significant portions of it. In DBLX we only use 2 api's that are specific to Java 1.6 and the rest are intentionally from 1.5 and 1.4 based on their performance.
- The use of any 3rd party libraries/jars infers you have some code to rewrite. Finding equivalent apis in Android for many 3rd party Java apis is difficult at best. It should be assumed that whatever your third-party library does you will likely have to recode it. The obvious solution is to include the third-party libraries with your Android application as it all runs under Java. But if you read below you will see the issues we had with native JAR support. It appears that true 'native jar' support is not part of Android.
- Threading code should use classic java threads: Anything other than the Java Thread does not work (or work the same) in Android. If you want your app to use Threads and have them work the same in Java and Andriod then you will need code that uses java.lang.Thread. If your Java app uses a thread library or object-management frameworks like Spring then it will not port to Android without numerous code changes.
- Make network IO pluggable: Its not trivial to have a client-server connections in Andriod. As a result, if your Java application has a TCP-IP listener or a port-based server component you will need a way to disable it completely when running in Android. Even if the code is compatible with Android APIs, the platform will require you to start your application and do some state investigation before you can turn up any network streams. Since DBLX is a multi-user client-server database, we needed a way to ensure that when in Andriod we were not using the network listener classes and threads at all.
- Make logging pluggable: While this is not clearly called out, the DBLX team has learned that its a bad practice to have a logger which writes out anywhere to the Android device. There are a million issues with logging on Android and they all have to do with the varying nature of the devices and the differences in types of storage and permissions. We even tried to use the built-in Java logger (java.util.Logging) since the apis are Android-compatible. But that had issues and only worked some of the time.
When we test and debug DBLX on Android we use a pluggable Java logger that outputs log data to stdout and stderr. When we are actually running on an Android device we actually perform no logging. We will revisit this when we have time, but for now it works the best.
- Make properties pluggable: If you have any properties for your Java app, like a properties XML file, you will want to make sure that the properties classes can be pluggable. The first issue is that your Android app will be packaged into an APK file. This causes the usual class loaders to behave different in Android than in Java proper. In Android, it means that your Java properties file will not be found without coding a class loader that knows where to find the XML properties in the APK file. Yuk.
Since the DBLX properties were mostly related to the paths where things are written, we use a properties file when running on Java, and we have a pluggable properties class that is injected into the database engine when running on Android.
- Expect to determine I/O paths with each application invocation: When using an Android device there will be times when the available file system has changed. Your application needs to be able to roll with these changes or it will not work very well. This is an issue for DBLX as we store off the database content in files. There are two primary file systems in an Android device, the internal storage and the external storage. Internal storage is present in all Android devices, but it can be very small and/or occupied by many versions of Android apps. External storage may be present in an Android device, but not all the time. Also, the size of external storage is all over the place and constantly changing, so you have to check for available space often to prevent application issues.
The worst part of I/O in Android is that if you have external storage, it can be disabled or rendered read-only by other apps and by connecting a data cable. You have to constantly check for available I/O locations and be ready to move things around if something changes.
The rough edges of Android development
The DBLX team used Eclipse for its Android development. If you check developer.android.com you will see that there are a ton of existing tools for Eclipse that make Android development fairly simple. The setup and installation documentation is excellent, and everything worked perfectly in Eclipse the first time.
But, we have had major issues with some of the 'features' of Android and would like to save you from having the same issues.
- JARs do not work: If you read about Android you will eventually find people having lots of problems with native Java JAR files. The Android documentation would lead you to believe you can include you Java JAR file with your Android code and it will all work.
It will not work and you will spend days trying to figure it out.
We tried to take the DBLX.jar file that we created for the DB server and use it as a JAR library in Android. All we got were a lot of classnotfound exceptions at run time. Oddly, since the jars are included in the classpath in Eclipse we never got compile errors, only runtime errors.
We found that you cannot use a Java JAR file in Android, even if it is 100% compatible with Android.
To create JAR files for our Android apps, we had to take the Java source code for DBLX and compile it in Eclipse as an Android project. Then we made a JAR file from the compiled classes and that jar worked properly in Android.
For DBLX this means that we have to manage two JAR files for the DB engine; one is the full DB engine compiled in Java 1.6 and the other is the same source code compiled against Android. We wanted one JAR that both could use, but we cannot get there at this time.
This means that when we get ready to release our Android libraries we will have two sets available - one for Android and one for Java 1.6 proper.
- Your UI problem is Gravity: If you create anything in Android, it is assumed you will have a user-interface. If you attempt to create a user interface you will find that in the beginning, every other UI widget you add does not display the way you expected or does not display at all. Android UI widgets have this concept of Gravity. It controls how a ui component is 'docked' relative to its parent.
When something in the UI does not display it is because you have the wrong Gravity settings. Gravity also has a concept of being a LayoutParam. LayoutParams are the code-equivalent of setting the Gravity based on an x-y coordinate. LayoutParams have to be set relative to their parents, and this is unclear from the Android documentation. The worst part about Gravity is that none of the concept makes any sense. When you find the Gravity property that is correct for your UI widget you will immediately be confused about why it worked.
It took the DBLX team 2 days to create a basic Android UI for DBLX and then it took us 30 days to get it working and looking the way we wanted. Now that we burned that 30 days and learned about how to use Gravity and LayoutParams we could achieve the same results in half the time.
- Debugging and the ARM emulator - Something sucks: To allow developers to test their Android code on different device configurations the Eclipse development tools come with an Android ARM emulator. This is like having a virtual machine image of an Android device that you can configure for all the phones and tablets you wish to test and support. The emulator is very cool and does work well.
But when you go to Debug your Android application in Eclipse it calls the ARM emulator and fires up an instance to run the Debug session.
On all our systems (Macs and PC's) we can only debug about 60% of the time. Many times the ARM emulator is fired up and the app is installed, but it does not connect to Eclipse and you never hit any breakpoints. This is one of the most frustrating aspects of Android development. If you are testing your app and there is an error, if the debugger is not connected and setup then your app force closes and you have no idea what happened. There were times where it took an hour to find a runtime exception, and had the debugger worked consistently we would have found the issue the first time the exception was thrown.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Porting to Android WTF: Lessons Learned
Porting DBLX to the Android platform turned out to be the most simple part of the exercise. It took minutes to port DBLX for use on Android, but then we had to spend weeks working out the packaging writing the UI components.
If you plan to port an application to Android, or create an Android application you may find some of the following WTF's very useful.
How will you know if your Java application can be ported to Andriod?
If your Java application uses JDK 1.6 and has few external dependencies, it will likely port to Android with little to no code changes required. The way you determine if your app will work is to create a new Andriod project in Eclipse and then import the source code for your application. Once the import is complete then Eclipse will attempt to compile everything, and if there is a compatability issue you will be notified immediately. If your Java app has dependencies on JAR files, then it is likely that you will have to port more than just your application to Android.
My Java app uses apis from JDK 1.7. Will it port to Andriod?
From what we can tell, you are fucked. Not only do JDK 1.7 apis not appear to be compatible with Andriod, we have had significant problems with JDK 1.7 in general. DBLX runs with Java apis which were in Java 1.6 and earlier. Java 1.6 seems to be where your application needs to be if you want to port to Andriod.
Is there anything you can do with your Java code to ensure it works seamlessly with Android?
We learned a few things about Java and Android that made the porting process extremely simple:
- Use pure Java Buffered IO streams: If you are using java.nio or using an external IO library you may not find an equivalent in the Android apis.
- Use org.w3c package references for all XML manipulation: When you use org.w3c packages for your XML parsing then Java uses the compatible XML library found in the classpath. So if you want to use Xerces as your XML parser you would use org.w3c package names in your code and make sure all the Xerces jar files are in your classpath.
By using the org.w3c packages for XML parsing you will get the same behavior between Java and Andriod and not have to change one line of code. Since DBLX is based on XML, this was a huge win for us.
- Use JDK 1.6 or earlier: Your app really needs to be completely Java 1.6 compatible to assume you won't have to rewrite significant portions of it. In DBLX we only use 2 api's that are specific to Java 1.6 and the rest are intentionally from 1.5 and 1.4 based on their performance.
- The use of any 3rd party libraries/jars infers you have some code to rewrite. Finding equivalent apis in Android for many 3rd party Java apis is difficult at best. It should be assumed that whatever your third-party library does you will likely have to recode it. The obvious solution is to include the third-party libraries with your Android application as it all runs under Java. But if you read below you will see the issues we had with native JAR support. It appears that true 'native jar' support is not part of Android.
- Threading code should use classic java threads: Anything other than the Java Thread does not work (or work the same) in Android. If you want your app to use Threads and have them work the same in Java and Andriod then you will need code that uses java.lang.Thread. If your Java app uses a thread library or object-management frameworks like Spring then it will not port to Android without numerous code changes.
- Make network IO pluggable: Its not trivial to have a client-server connections in Andriod. As a result, if your Java application has a TCP-IP listener or a port-based server component you will need a way to disable it completely when running in Android. Even if the code is compatible with Android APIs, the platform will require you to start your application and do some state investigation before you can turn up any network streams. Since DBLX is a multi-user client-server database, we needed a way to ensure that when in Andriod we were not using the network listener classes and threads at all.
- Make logging pluggable: While this is not clearly called out, the DBLX team has learned that its a bad practice to have a logger which writes out anywhere to the Android device. There are a million issues with logging on Android and they all have to do with the varying nature of the devices and the differences in types of storage and permissions. We even tried to use the built-in Java logger (java.util.Logging) since the apis are Android-compatible. But that had issues and only worked some of the time.
When we test and debug DBLX on Android we use a pluggable Java logger that outputs log data to stdout and stderr. When we are actually running on an Android device we actually perform no logging. We will revisit this when we have time, but for now it works the best.
- Make properties pluggable: If you have any properties for your Java app, like a properties XML file, you will want to make sure that the properties classes can be pluggable. The first issue is that your Android app will be packaged into an APK file. This causes the usual class loaders to behave different in Android than in Java proper. In Android, it means that your Java properties file will not be found without coding a class loader that knows where to find the XML properties in the APK file. Yuk.
Since the DBLX properties were mostly related to the paths where things are written, we use a properties file when running on Java, and we have a pluggable properties class that is injected into the database engine when running on Android.
- Expect to determine I/O paths with each application invocation: When using an Android device there will be times when the available file system has changed. Your application needs to be able to roll with these changes or it will not work very well. This is an issue for DBLX as we store off the database content in files. There are two primary file systems in an Android device, the internal storage and the external storage. Internal storage is present in all Android devices, but it can be very small and/or occupied by many versions of Android apps. External storage may be present in an Android device, but not all the time. Also, the size of external storage is all over the place and constantly changing, so you have to check for available space often to prevent application issues.
The worst part of I/O in Android is that if you have external storage, it can be disabled or rendered read-only by other apps and by connecting a data cable. You have to constantly check for available I/O locations and be ready to move things around if something changes.
The rough edges of Android development
The DBLX team used Eclipse for its Android development. If you check developer.android.com you will see that there are a ton of existing tools for Eclipse that make Android development fairly simple. The setup and installation documentation is excellent, and everything worked perfectly in Eclipse the first time.
But, we have had major issues with some of the 'features' of Android and would like to save you from having the same issues.
- JARs do not work: If you read about Android you will eventually find people having lots of problems with native Java JAR files. The Android documentation would lead you to believe you can include you Java JAR file with your Android code and it will all work.
It will not work and you will spend days trying to figure it out.
We tried to take the DBLX.jar file that we created for the DB server and use it as a JAR library in Android. All we got were a lot of classnotfound exceptions at run time. Oddly, since the jars are included in the classpath in Eclipse we never got compile errors, only runtime errors.
We found that you cannot use a Java JAR file in Android, even if it is 100% compatible with Android.
To create JAR files for our Android apps, we had to take the Java source code for DBLX and compile it in Eclipse as an Android project. Then we made a JAR file from the compiled classes and that jar worked properly in Android.
For DBLX this means that we have to manage two JAR files for the DB engine; one is the full DB engine compiled in Java 1.6 and the other is the same source code compiled against Android. We wanted one JAR that both could use, but we cannot get there at this time.
This means that when we get ready to release our Android libraries we will have two sets available - one for Android and one for Java 1.6 proper.
- Your UI problem is Gravity: If you create anything in Android, it is assumed you will have a user-interface. If you attempt to create a user interface you will find that in the beginning, every other UI widget you add does not display the way you expected or does not display at all. Android UI widgets have this concept of Gravity. It controls how a ui component is 'docked' relative to its parent.
When something in the UI does not display it is because you have the wrong Gravity settings. Gravity also has a concept of being a LayoutParam. LayoutParams are the code-equivalent of setting the Gravity based on an x-y coordinate. LayoutParams have to be set relative to their parents, and this is unclear from the Android documentation. The worst part about Gravity is that none of the concept makes any sense. When you find the Gravity property that is correct for your UI widget you will immediately be confused about why it worked.
It took the DBLX team 2 days to create a basic Android UI for DBLX and then it took us 30 days to get it working and looking the way we wanted. Now that we burned that 30 days and learned about how to use Gravity and LayoutParams we could achieve the same results in half the time.
- Debugging and the ARM emulator - Something sucks: To allow developers to test their Android code on different device configurations the Eclipse development tools come with an Android ARM emulator. This is like having a virtual machine image of an Android device that you can configure for all the phones and tablets you wish to test and support. The emulator is very cool and does work well.
But when you go to Debug your Android application in Eclipse it calls the ARM emulator and fires up an instance to run the Debug session.
On all our systems (Macs and PC's) we can only debug about 60% of the time. Many times the ARM emulator is fired up and the app is installed, but it does not connect to Eclipse and you never hit any breakpoints. This is one of the most frustrating aspects of Android development. If you are testing your app and there is an error, if the debugger is not connected and setup then your app force closes and you have no idea what happened. There were times where it took an hour to find a runtime exception, and had the debugger worked consistently we would have found the issue the first time the exception was thrown.