Now the next challenge was to convert the pdf I am interested in to the format supported by Kindle app(.mobi or .azw). So found this online converter. It will convert .pdf to .mobi . Move this .mobi into your /kindle directory and you will found the book in your Kindle bookshelf.
Happy reading !!

Till now we have seen

1. What are the issues of running CXF on GAE ?
   
2. How to run JAXB on GAE ?

In this post we will see how will we run make changes in Apache CXF source so that everything starts working.

The ideas and changes are influenced by one of the apache CXF developer Daniel Kulp who gave some interesting insights about CXF.

Issues:

1. Writing to file should be disabled as it is restricted by GAE
2. GAE doesn’t allow to spawn new threads
3. Changes in JAXB source should be integrated with CXF

Changes are done keeping in mind that

• We are interested only in JAX-RS
• WSDL is not required

CXF has “workqueues” using threads which provides support for one-way operations. But as in JAX-RS we hardly have any one way operations so we need not consider about “workqueues.

For attachments and message logging, a special output stream is used such that once a threshold is reached, it starts saving to temporary files on the file system.  As on GAE we can’t create new files. So we have two options

• To save all that data which was earlier written by CXF in a temp file to GAE datastore.
• We can just proceed keeping them in memory.

But the, the maximum message size GAE allows in or out is 10MB(which also sometimes cause a problem as some kind of “Server Error” starts coming  on sending a request of ≥1MB and so we are only able to send a request of ≤1MB successfully ) Thus, the message sizes are small enough to keep them in memory itself. So we will proceed by disabling this feature of saving to temp files.

We also have to reflect the changes in JAXB (i.e. to add jaxb.jar) and change the classes to use the new packages(changed names) from jaxb.jar instead of jaxb-impl*.jar and jaxb-api*.jar

Changes:

1.  In  org.apache.cxf.io.CachedOutputStream.java

Comment out regions using FleOutputStream , boolean inmem and threshold

After commenting out these things CXF won’t make any temporary files for storing logs or attachments.

2.  Change the package name from javax.xml.bind.* to ae.javax.xml.bind.* in the following files :-

• org.apache.cxf.jaxrs.provider.XSLTJaxbProvider.java
• org.apache.cxf.ws.policy.builder.jaxb.JaxbAssertionBuilder.java
  • org.apache.cxf.transport.http.policy.HTTPClientAssertionBuilder.java
  • org.apache.cxf.transport.http.policy.HTTPServerAssertionBuilder.java
  • org.apache.cxf.transport.AbstractMultiplexDestination.java
  • org.apache.cxf.jaxrs.provider. AbstractJAXBProvider.java

3. Disable saving to file in

• org.apache.cxf.interceptor.LoggingInInterceptor.java by commenting out

if (bos.getTempFile() != null) {

//large thing on disk…

buffer.getMessage().append(“\nMessage (saved to tmp file):\n”);

buffer.getMessage().append(“Filename: ” + bos.getTempFile().getAbsolutePath() + “\n”);

}

• org.apache.cxf.interceptor.LoggingOutInterceptor.java by commenting out

if (cos.getTempFile() == null) {

//buffer.append(“Outbound Message:\n”);

if (cos.size() > limit) {

buffer.getMessage().append(“(message truncated to ” + limit + ” bytes)\n”);

}

}else {

buffer.getMessage().append(“Outbound Message (saved to tmp file):\n”);

buffer.getMessage().append(“Filename: ” + cos.getTempFile().getAbsolutePath() + “\n”);

if (cos.size() > limit) {

buffer.getMessage().append(“(message truncated to ” + limit + ” bytes)\n”);

}

}

4. In the file org.apache.cxf.jaxrs.provider.JAXBElementProvider.java

Change the package name from javax.xml.bind.* to ae.javax.xml.bind.*

Change the function as

protected Object unmarshalFromReader(Unmarshaller unmarshaller, XMLStreamReader reader, MediaType mt) throws JAXBException {

return unmarshaller.unmarshal ((ae.javax.xml.stream.XMLStreamReader)reader);

}

public Object readFrom(Class<Object> type, Type genericType, Annotation[] anns, MediaType mt,

MultivaluedMap<String, String> headers, InputStream is)

throws IOException {

try {

Class<?> theType = getActualType(type, genericType, anns);

Unmarshaller unmarshaller = (Unmarshaller)createUnmarshaller(theType, genericType);

……..

……..

……..

The changes done are normal typecasting of reader and unmarshaller.

With these changes I got my CXF-RS app running on GAE.

A sample CXF-RS app on GAE can be viewed on appengine

This has been discussed in one of the Google appengine developer forums and I think now GAE developers have included JAXB in  JRE whitelist too. But below I am going to tell the changes I have done to get my JAXB running on GAE (though it is not required for most of you as now there’s is a support already but for the interested ones below are the changes I had done)

Issues :

1. FileOutputStream in com.sun.xml.bind.v2.runtime.MarshallerImpl and javax.xml.bind.helpers.AbstractMarshallerImpl
2. use of AWT in com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl
3. java.util.logging.ConsoleHandler, which is also not supported by Google App

Engine’s Java runtime environment, but used in javax.xml.bind.ContextFinder

Steps :

1. Download JAXB source code from

https://jaxb.dev.java.net/2.1.11/jaxb-ri-2_1_11.src.zip

Put it in an Eclipse project at ~/workspace/jaxb-ri-2_1_11

2.    Build the runtime as we need to do this so that generated source gets generated.

3.    In com.sun.xml.bind.v2.model.impl.ReferencePropertyInfoImpl.java

Change

boolean isRequired(XmlElementRef ref)

to always return true.

As it deals with something related to compatibility issues in 2.2 API.

The same issue stops xjc subproject from building, but we don’t care about that.

To avoid that I changed the compile task in build.xml to read:

<target

depends=”init-standalone”

description=”compile runtime, and xjc sub-projects”>

<ant dir=”apgen”               target=”compile” />

<ant dir=”runtime”             target=”compile” />

<ant dir=”runtime-fi”          target=”compile” />

<ant dir=”runtime-staxex”      target=”compile” />

<ant dir=”runtime-deprecated”  target=”compile” />

<!–    <ant dir=”xjc”                 target=”compile” />

<ant dir=”tools”               target=”compile” /> –>

</target>

4.    Copy runtime/src and runtime/build/src to your GAE project [here called myproj] src dir

runtime/src$ cp com /home/dev/workspace/myproj/src/ -rR

runtime/build/src$ cp * /home/dev/workspace/myproj/src/ -rR

5. Find jaxb-api-src.zip in tools/lib/redist. It contains javax.xml.bind

Unzip that, and put it your myproject src dir

6.    Find jsr173_1.0_src.jar; it is in tools/lib/src.  This contains javax.xml.stream.XMLStreamWriter and XMLStreamException which we need but aren’t whitelisted.

Extract them and copy to myproject.

src tree should now include

/com/sun/xml/bind

/javax/xml/bind

/javax/xml/stream

7.    Add the missing jars

cp tools/lib/redist/jsr173_1.0_api.jar

/home/dev/workspace/myproj/war/WEB-INF/lib/

cp tools/lib/redist/activation.jar /home/dev/workspace/myproj/war/WEB-INF/lib/

cp tools/lib/rebundle/runtime2/*.jar /home/dev/workspace/myproj/war/WEB-INF/lib/

8.    Now fix the major issues :-

• com.sun.xml.bind.v2.runtime.MarshallerImpl

Comment out the FileOutputStream part as we won’t have file for marshalling in our case

• javax.xml.bind.helpers.AbstractMarshallerImpl

Comment out

void marshal(Object jaxbElement, File output)

as we won’t be dealing with marshalling of files and also its not supported J

• com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl.java

As AWT is not supported so we have to comment out

PcdataImpl<Image>(Image.class, createXS(“base64Binary”))

convertToBufferedImage(Image image)

Base64Data print(Image v)

• ae.javax.xml.bind.ContextFinder.java

As java.util.logging.ConsoleHandler is not supported comment out the logger part

9.    Exclude the classes  from validation (right click > Google > App Engine Settings >App Engine > Validation).  These are

• com.sun.xml.bind.v2.runtime.property.SingleMapNodeProperty

• com.sun.xml.bind.v2.util.TypeCast

Interestingly, the validator wonn’t complain about javax.xml.bind, now that it is in our source tree. And it’s enough for the app to start in the Jetty simulation environment.

But it will try to use the JAXB in Java 6, not the ones in my source tree.  So ename:

• com.sun.xml.bind –> ae.com.sun.xml.bind

• javax.xml.bind   –> ae.javax.xml.bind

• javax.xml.stream   –> ae.javax.xml.stream

For JAXBContext: ae.com.sun.xml.bind.v2.runtime.JAXBContextImpl

For marshaller: ae.com.sun.xml.bind.v2.runtime.MarshallerImpl

10. Finally in ae.com.sun.xml.bind.v2.runtime.reflect.opt.Injector.java

Comment out

AccessController.doPrivileged(new PrivilegedAction<Void>() {

public Void run() {

defineClass.setAccessible(true);

resolveClass.setAccessible(true);

return null;

}

});

As it will cause create some security related issues.

This will make JAXB running on GAE.

Reference:

http://code.google.com/p/googleappengine/issues/detail?id=1267

At the end of the project, I with help of my mentors Prasad Kumbhare and Mahesh Sankaran were able to host CXF-RS app on GAE.

Basically what  we were interested in doing is to provide support for CXF Restful applications on GAE

After browsing through web we made our requirements clear, we understood that

With the launch of Google App Engine, web applications are now very easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it’s ready to serve your users.

CXF is one of the most popular web services framework used currently to develop web services. So a lot of people today are asking for CXF support on GAE. At present, GAE does not support CXF and our goal is to fulfil this requirement.

CXF-RS Support on GAE

Following issues were identified with CXF that prevent it to run on GAE:

1.      CXF and JAXB use file IO at some places for logging purpose which is restricted on GAE

2.      CXF spawns threads for one way operations. Threads are not supported on GAE.

3.      CXF uses JAXB for xml processing. Changes in JAXB source should be integrated with CXF

To use CXF services on GAE, follow the steps below:

1.    Create   GAE project using eclipse plug-in provided by Google.

2.    Add following jars to your class path and also copy them to your projects WEB-INF/lib folder.

1. CXF Jars from <CSF_ROOT>\lib folder.
2. Modified CXf.jar and jaxb.jar

3.    Modify web.xml to use CXF-Servlet for handling web requests.

4.    Copy CXF-Servelt.xml from CXF Installation to WEB-INF folder.

So after identifying the issues we first have to make sure that we resolve these issues to get our Apache CXF-RS app running on GAE . In future posts I will tell about the changes I had done to get it running.