Friday, 4 November 2011

Could not complete schema update

I recently mapped a JPA entity to a view, and encountered this error while using hibernate to update the database schema:
ORA-01702: a view is not appropriate here

[SchemaUpdate.execute] could not complete schema update

I was using the Oracle JDBC drivers 10.1.3.3 - luckily, the solution was to upgrade the drivers - when using 11.2.0.2.0 it works fine.
Download ojdbc6.jar from here:




ojdbc6.jar (2,152,051 bytes) - Classes for use with JDK 1.6. It contains the JDBC driver classes except classes for NLS support in Oracle Object and Collection types.

Monday, 17 October 2011

On to some exciting stuff

I went to a Agile Sydney meetup recently - "Automated web tests as a team communication tool" - by John Smart of Wakaleo Consulting. It was great stuff, and the presentation is now online.

There is a nice story here in that the project feature set is documented via
easily readable and executable tests, and the reporting shows which features are
implemented and which aren't.

More information on the the Thucydides tool he references can be found at:

While reading that presentation online, I found more gold here in some of his other presentations:

They are a great read and are great pointers in the right direction to improve your software development.

(If you haven't seen them before, check out Wakaleo's training courses. I haven't been on them, but based on the course content, they look to be the most relevant courses I've come across)

The last couple of projects I've been on I've been frustrated by a few things about the code base that I couldn't quite put my finger on, and the DDD presentation hit a nerve.

I searched around for more about DDD and found some more excellent resources:

I still have to absorb all of this material and try it out but I can instantly recognise that it addresses some if not most of the code problems that have frustrated me.

So, exciting times are ahead. Its always refreshing to find new tricks and efficient ways of improving software development!

Friday, 16 September 2011

Using the Crystal Reports Java API to generate PDF

I recently had to investigate how to generate a PDF from a Crystal Report created by another team. Without knowing anything about Crystal Reports, I had to google around for information and piece it all together. It turns out to be really simple once you know how.

We were using Business Objects 4.0, and probably the most important thing was to get the Java library - download 'SAP Crystal Reports for Java runtime components - Java Reporting Component (JRC)' from
http://www.businessobjects.com/campaigns/forms/downloads/crystal/eclipse/datasave.asp

There are a lot of samples on the web to look at - you might find something to help here:
http://wiki.sdn.sap.com/wiki/display/BOBJ/Java+Reporting+Component++SDK+Samples

A good example to start with is “JRC EXPORT REPORT”:
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/40d580ce-bd66-2b10-95b0-cc4d3f2dcaef

If you have an RPT file, the java to generate the report is relatively simple - :

ReportClientDocument reportClientDoc = new ReportClientDocument();
reportClientDoc.open("My crystal report.rpt", 0);
ByteArrayInputStream byteArrayInputStream = (ByteArrayInputStream)reportClientDoc.getPrintOutputController().export(ReportExportFormat.PDF);
reportClientDoc.close();

The report I dealt with was developed by another team, and expected a data source called ‘TESTDB’ to be available. Without that data source present, I would get the following error:

com.crystaldecisions.sdk.occa.report.lib.ReportSDKException: Error finding JNDI name (TESTDB)---- Error code:-2147467259 Error code name:failed


Setting a data source up in Tomcat is trivial, just by adding a resource to context.xml:

<Context>
<WatchedResource>WEB-INF/web.xml</WatchedResource>

<Resource name="jdbc/TESTDB" auth="Container" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000"
username="user" password="passwd" driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:@dbserver:1521:db1"/>
</Context>

Now the data source is set up, there’s another problem - the report expects a parameter and we get this error:

com.crystaldecisions.sdk.occa.report.lib.ReportSDKParameterFieldException: InternalFormatterException---- Error code:-2147217394 Error code name:missingParameterValueError


Adding the report parameter is simple once you know how:

ParameterFieldController paramController = reportClientDoc.getDataDefController().getParameterFieldController();
paramController.setCurrentValue("","MyParamName","MyParamValue");

There’s probably a lot of subtle information and detail missing here, but it works - I can generate the PDF via java code so its now at a stage where it can be integrated into our application.

So, the final spike test code looks like:

<%@page contentType="text/html"%>
<%@page pageEncoding="UTF-8"%>

<%//Crystal Java Reporting Component (JRC) imports.%>
<%@page import="com.crystaldecisions.reports.sdk.*" %>
<%@page import="com.crystaldecisions.sdk.occa.report.lib.*" %>
<%@page import="com.crystaldecisions.sdk.occa.report.exportoptions.*" %>

<%//Java imports. %>
<%@page import="java.io.*" %>

<%

try {

//Open report.
ReportClientDocument reportClientDoc = new ReportClientDocument();
reportClientDoc.open("MyReport.rpt", 0);
ParameterFieldController paramController = reportClientDoc.getDataDefController().getParameterFieldController();
paramController.setCurrentValue("","MyParamName","MyParamValue");
ByteArrayInputStream byteArrayInputStream = (ByteArrayInputStream)reportClientDoc.getPrintOutputController().export(ReportExportFormat.PDF);
reportClientDoc.close();

writeToBrowser(byteArrayInputStream, response, "application/pdf");

} catch(Exception ex) {
out.println(ex);
}
%>

<%!
/*
* Utility method that demonstrates how to write an input stream to the server's local file system.
*/
private void writeToBrowser(ByteArrayInputStream byteArrayInputStream, HttpServletResponse response, String mimetype) throws Exception {

//Create a byte[] the same size as the exported ByteArrayInputStream.
byte[] buffer = new byte[byteArrayInputStream.available()];
int bytesRead = 0;

//Set response headers to indicate mime type and inline file.
response.reset();
response.setHeader("Content-disposition", "inline;filename=report.pdf");
response.setContentType(mimetype);

//Stream the byte array to the client.
while((bytesRead = byteArrayInputStream.read(buffer)) != -1) {
response.getOutputStream().write(buffer, 0, bytesRead);
}

//Flush and close the output stream.
response.getOutputStream().flush();
response.getOutputStream().close();

}
%>

Note, implementing this in a JSP was just a simple and quick shortcut to investigate the API.

I put a simple CRConfig.xml in WEB-INF/classes:

<?xml version="1.0" encoding="utf-8"?>
<CrystalReportEngine-configuration>
<reportlocation>..</reportlocation>
<timeout>0</timeout>
<ExternalFunctionLibraryClassNames>
<classname></classname>
</ExternalFunctionLibraryClassNames>
</CrystalReportEngine-configuration>


Note that this specified a report location of .. which meant I had to put MyReport.rpt in the WEB-INF directory of my application (i.e. [web-root]/WEB-INF/classes/.. = [web-root]/WEB-INF) - contents of the WEB-INF directory should not be available for download.

If the rpt file cannot be found, you’ll see an error like:

com.crystaldecisions.sdk.occa.report.lib.ReportSDKException: Report file /[path-to-webapps]/webapps/cr/WEB-INF/MyReport.rpt not found---- Error code:-2147215356 Error code name:fileNotOpened

Monday, 12 September 2011

Easy classpaths with Java 6

Back in the day it used to be a bit more difficult than it is now to dynamically generate your java application classpath - you'd have to write a script to loop over all the jar files in a directory, appending them to your classpath variable - like this.

Since Java 6 though, its been a lot easier - you can use wildcards to specify all jars in a directory. See the "Understanding class path wildcards" in the Java SE 6 documentation.

Now its as simple as:

java -cp /my/lib/dir/* MyClass


Simple! As it should be!

Saturday, 10 September 2011

Turning off JDK logging

I've been developing a simple little command line application and one of the libraries I'm using seems to log debug information to the console. To clean up the console output, I had to turn off the JDK by using:
        LogManager.getLogManager().reset();
This seemed to do the trick for me, but it sounds like in some cases you may need to go a bit further and turn off logging at the global logger level:
        LogManager.getLogManager().reset();
        Logger globalLogger = Logger.getLogger(java.util.logging.Logger.GLOBAL_LOGGER_NAME);
        globalLogger.setLevel(java.util.logging.Level.OFF);

Friday, 9 September 2011

JUnit parameterized test with Spring autowiring AND transactions

I've been writing JUnit Parameterized tests (a nice intro here - also see Theories) to do some data driven API testing. Now, when introducing Spring into the mix there are a couple of extra things to do. I came unstuck though because I was trying to do Transactional tests - since I'm now using @RunWith(Parameterized.class) and setting up my Spring TestContextManager manually the @Transaction annotations caused an exception:
java.lang.IllegalArgumentException: Superclass has no null constructors but no arguments were given


I couldn't find any built in solution, so I've gone with manual transaction management in my test, using doInTransaction:

@RunWith(Parameterized.class)
@ContextConfiguration(locations = "classpath*:/testContext.xml")
public class MyTest {

@Autowired
PlatformTransactionManager transactionManager;

private TestContextManager testContextManager;

public MyTest (... parameters for test) {
// store parameters in instance variables
}

@Before
public void setUpSpringContext() throws Exception {
testContextManager = new TestContextManager(getClass());
testContextManager.prepareTestInstance(this);
}

@Parameterized.Parameters
public static Collection generateData() throws Exception {
ArrayList list = new ArrayList();
// add data for each test here
return list;
}

@Test
public void validDataShouldLoadFully() throws Exception {
new TransactionTemplate(transactionManager).execute(new TransactionCallback() {
public Object doInTransaction(TransactionStatus status) {
status.setRollbackOnly();
try {
... do cool stuff here

} catch (Exception e) {
throw new RuntimeException(e);
}
return null;
}
});

}

Saturday, 20 August 2011

Week in Review - 2011-33

I've occasionally been frustrated by the output of 'ps -ef' not showing the full command line for the processes. Well, it turns out that the solution is as simple as piping the output to a file - i.e. 'ps -ef > ps.txt' - now I can easily see the FULL command line for my java processes.

I bought a new WIFI router this week. My old one was getting flaky, and for some reason I've always had trouble with some of the url shorteners such as bit.ly and t.co - they just couldn't be resolved. So I bought a basic Netgear WNR1000 which satisfied all of my criteria. One of the things I like most is that it has a switch to turn off the wireless when I don't need it. This helps with security as well as saving power.

I've almost finished porting my latest application to AppEngine (Python). Lucky its a small application! I'm constantly being reminded that its useful to have skills in a lightweight technology to bootstrap and test ideas quickly. If that idea takes off, then there is always the option to re-write in a more heavy weight enterprise way. Using different languages and technologies can also be refreshing and stimulating, but it does take time.

Saturday, 13 August 2011

Week in Review 2011-32

This post is on the money. Humorous and accurate. I've seen this all before.

There is a very interesting podcast on the software patent situation at NPR Planet Money.

I'm in a bind! I'm ready to launch a new app (Grails based), but my VPS with 768MB memory can't handle it (its running vsconsole already, and apache, mysql etc). Unfortunately I can't add more memory until next month! They are out of stock!! I signed up for CloudFoundry.com (still in BETA), but it I haven't been able to find out how to use my own domain, or how to extract the data when/if I need to. I could easily port it to AppEngine (Python). It's a small app so it wouldn't be much effort but I'd rather spend the effort on new features!

Tuesday, 9 August 2011

Week in Review - 2011-31

I am currently working on a JSF2 application at work, and I mostly use Firefox during the development process because I use SeleniumIDE as a development/productivity tool to quickly log in as different users and drive the browser to a certain point. Imagine my surprise when I found out that something didn't work in IE8. I could see IE reporting a JavaScript error, but the details were useless. Even with the developer tools in IE, I couldn't identify what or where the error was. This page didn't have any JavaScript written by me - its all the JSF generated code. All I could do is use the process of elimination and trial and error to rearrange the page and subtlety change code  until it worked. Its hard to believe that IE is still in this state, especially when faced with the competition from Chrome (built in developer tools) and Firefox (add-on FireBug for developer tools).

I'm using VMWare Fusion on my MacBook AIR to run an Ubuntu server for testing purposes. I've set the network to NAT so I can SSH to it and administer it. I've shared folders on the MacBook host with the Ubuntu guest (available in /mnt/hgfs/). Now all I need to do is set up the software so I can perform the tests I need.

Friday, 29 July 2011

Week in Review - 2011-30

A friend pointed out this article to me which says in summary:
"No matter how much you try, you can’t stop people from sticking beans up their nose."

Its well worth reading, especially if you continually see crazy things happening at work, and need a way to deal with it.

In a similar fashion, I've also stumbled across a couple of comics which happen to describe (too accurately) most of the real world: OneFTE.com and PhD Comics. And lets not forget Dilbert. At least we can laugh about it.

I've been having trouble with my MythTV install. I've installed 2 tuner cards (one is a single tuner, the other dual) and when I reboot they seem to be randomly assigned to /dev/dvb/adapterX - I finally started looking into it, and found this reference which has helped me permanently assign the first card - which should be consistent across reboots. However, I've still got to set up the second card, and test properly.

An article (from 2007!) pointing out how Ron Jefferies failed to build a sudoko solver using TDD sparked a very interesting conversation in which we debated the usefulness of tests. It also sparked an interesting tangent, because it appears that most people think the second D in TDD stands for design (it actually stands for DEVELOPMENT). It also seems to be thought that if you do TDD you don't need to do any application design (a feeling that maybe Ron Jefferies sudoko posts perhaps promoted). I certainly prefer to work on a project where the high level design is mapped out and everyone understands (for example, mapping out the subsystems and how they interact). I like tests and I will continue to write them - I believe it improves the design of my CLASS, test-first where appropriate, but I'll admit that its not always easy. It takes a bit of skill to write good tests, but I think developing this skill helps improve your production code as well. Its a state of mind, and spirit+passion is important.

I currently have an old pre-paid phone. I use an iPod touch to listen to podcasts/music and play the odd game. It would be nice to shiny new Android smart phone so I could do this all on one device. So, looking around I was stunned to see so many phones running Android 2.1 and 2.2!! It seems like there are not many 2.3 devices out there. If I did get a new phone, I'd want to tether my laptop - so the minimum Android would be 2.2. I spend $150 p/y on my pre-paid phone, and theres another $150 p/y if I use my prepaid 3G broadband modem for internet access on the go. So the Garmin A50 on a $19 plan looks competitive but there is no indication of the Android version. I think that's done on purpose to make it hard to compare and to generally frustrate customers ;-). The next feasible ones would be on the $29 plan if they ran 2.3 but alas: LG Optimus (unknown version) and Samsung Galaxy S (2.1). Sigh, too hard. Anyone know why these phones are crippled with old software?

Thursday, 7 April 2011

Viewing server log files

I think viewing server logs has always been more difficult than it should have been for me.

For example, when I'm developing and debugging applications deployed to the dev and test servers, if you want to see the various log files involved it requires:
  • Logging in to the server (if you have an account)

  • Finding the log files (if you remember where they are and have permissions to read them)

  • Tailing in a console
Some places its not trivial to get accounts and permissions - but anyway, the point is it takes more time than its worth to find that log file and see whats going on.

To make it easier, I created vsConsole - a java (grails) based web application that polls agents on the servers - so with a couple of clicks you can be tailing your favourite log file on your dev/test server.

It works well for our testers that I work with on my day job - it saves them the hassle of ssh and unix accounts and permissions. The convenience of a browser based application is in their opinion worthwhile and they get real value from it.

Select a file and when the page loads, you can toggle tailing by clicking the "Start/Stop tailing" button - and you'll see updates file gets appended to.

For more information, see
You can download it and try for yourself - it works best with Tomcat and Chrome.

I'd appreciate feedback - I'm considering building version 2 with a desktop like UI and other features if there is any interest.