lizongbo at 618119.com 工作,生活,Android,前端,Linode,Ubuntu,nginx,java,apache,tomcat,Resin,mina,Hessian,XMPP,RPC

2009年04月12日

Windows下编写google app engine jsp出现中文乱码问题的真正原因

Filed under: Java — 标签:, , , , — lizongbo @ 12:54

Windows下编写google app engine jsp出现中文乱码问题的真正原因,是com.google.appengine.tools.admin.Application里启动javac没加-encoding参数导致.

在Linux下,因为一般都是LANG=zh_CN.UTF-8,因此就不会遇到这个问题了。

Windows下一般是取的默认的file.encoding为GBK,因此根据jsp生成的class文件就是乱码内容了。

验证流程如下:
1.首先重命名E:\Java\appengine-java-sdk-1.2.0\lib\shared\jsp\jasper-compiler-5.0.28.jar为 jasper-compiler-5.0.28.jar.bak。
然后在E:\Java\appengine-java-sdk-1.2.0\bin下运行 appcfg update E:\Java\workspace\testwe\war。
这个时候在命令行下会看到如下的出错信息。
E:\Java\appengine-java-sdk-1.2.0\bin>java -cp “E:\Java\appengine-java-sdk-1.2.0\bin\\..\lib\appengine-tools-api.jar” com.google.appengine.tools.admin.AppCfg update e:\Java\workspace\testweb\war
Reading application configuration data…
2009-04-12 03:05:34.240::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
Beginning server interaction for lizongbo…
0% Creating staging directory
5% Scanning for jsp files.
8% Compiling jsp files.
Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/jasper/JspC
at com.google.appengine.tools.development.LocalJspC.main(LocalJspC.java:
14)
Caused by: java.lang.ClassNotFoundException: org.apache.jasper.JspC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
… 1 more
java.lang.RuntimeException: Failed to compile jsp files via E:\Java\jdk1.6.0_13\
jre\bin\java.exe -classpath com.google.appengine.tools.development.LocalJspC -uriroot C:\DOCUME
~1\ADMINI~1\LOCALS~1\Temp\appcfg8338201102819535049.tmp -p org.apache.jsp -l -v
-webinc C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\appcfg8338201102819535049.tmp\WEB-INF
\generated_web.xml -d C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\appcfg83382011028195350
49.tmp\WEB-INF\classes

2.将jasper-compiler-5.0.28.jar.bak重命名回 jasper-compiler-5.0.28.jar,
将E:\Java\jdk1.6.0_13\bin\javac.exe重命名为 javacc.exe,
在Eclipse里通过Google app Engine的插件发布,得到下面的出错信息:

Unable to upload:
java.lang.IllegalStateException: cannot find javac executable based on java.home, tried “E:\Java\jdk1.6.0_13\jre\bin\javac.exe” and “E:\Java\jdk1.6.0_13\bin\javac.exe”
at com.google.appengine.tools.admin.AppAdminFactory$ApplicationProcessingOptions.getJavaCompiler(AppAdminFactory.java:325)
at com.google.appengine.tools.admin.Application.compileJavaFiles(Application.java:340)
at com.google.appengine.tools.admin.Application.compileJsps(Application.java:326)
at com.google.appengine.tools.admin.Application.createStagingDirectory(Application.java:235)
at com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:39)
at com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngineBridgeImpl.java:203)
at com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(DeployProjectJob.java:97)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
java.lang.IllegalStateException: cannot find javac executable based on java.home, tried “E:\Java\jdk1.6.0_13\jre\bin\javac.exe” and “E:\Java\jdk1.6.0_13\bin\javac.exe”
于是根据堆栈找到了com.google.appengine.tools.admin.Application,使用Jd-gui(来自http://java.decompiler.free.fr/)打开class找到了如下代码:
[code]
String javacCmd = opts.getJavaCompiler().getPath();
args.add(javacCmd);
args.add(“-classpath”);
args.add(classpath.toString());
args.add(“-d”);
args.add(classDir.getPath());

for (File f : new FileIterator(classDir)) {
if (f.getPath().toLowerCase().endsWith(“.java”))
args.add(f.getPath());

}

if (args.size() == 5)
return;

Process javac = startProcess((String[])args.toArray(new String[0]));
[/code]

从代码看到,Google App engine只用 apache Tomcat jasper生成java文件,再通过拼凑命令行字符串方式生成编译java文件为Class的命令,通过Process执行。
由于命令行参数没有设置encoding参数,于是javac使用了系统默认的file.encoding,在中文Windows下,基本都是GBK。在Linux下,因为一般都是LANG=zh_CN.UTF-8,因此就不会遇到这个问题了。
下面是linux 下ps看到的完整javac命令:

lizongbo  6733  0.0  0.4 690156  9228 ?        Sl   21:16   0:00 /usr/local/jdk1.6.0_12/bin/javac -classpath /usr/local/java/appengine-java-sdk-1.2.0/lib/impl/appengine-api.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/impl/appengine-api-stubs.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/impl/appengine-local-runtime.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/geronimo-el_1.0_spec-1.0.1.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/geronimo-servlet_2.5_spec-1.2.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/appengine-local-runtime-shared.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/geronimo-jsp_2.1_spec-1.0.1.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/jasper-runtime-5.0.28.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/commons-el-1.0.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/ant-launcher-1.6.5.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/commons-logging-1.1.1.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/ant-1.6.5.jar:/usr/local/java/appengine-java-sdk-1.2.0/lib/shared/jsp/jasper-compiler-5.0.28.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/classes:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/ant-launcher-1.6.5.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/jakarta-standard-1.1.2.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/jasper-compiler-5.0.28.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/datanucleus-appengine-1.0.0.final.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/ant-1.6.5.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/datanucleus-core-1.1.0.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/geronimo-jta_1.1_spec-1.1.1.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/jdo2-api-2.3-SNAPSHOT.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/jasper-runtime-5.0.28.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/commons-logging-1.1.1.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/appengine-api-1.0-sdk-1.2.0.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/geronimo-jpa_3.0_spec-1.1.1.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/datanucleus-jpa-1.1.0.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/commons-el-1.0.jar:/tmp/appcfg8102867646621343748.tmp/WEB-INF/lib/jakarta-jstl-1.1.2.jar: -d /tmp/appcfg8102867646621343748.tmp/WEB-INF/classes /tmp/appcfg8102867646621343748.tmp/WEB-INF/classes/org/apache/jsp/test_jsp.java /tmp/appcfg8102867646621343748.tmp/WEB-INF/classes/org/apache/jsp/p_jsp.java

在Google app Engine的java SDK没有完善之前,只有在Ubuntu下写jsp最方便不会出现中文乱码问题。
受网上文章相关文章的误导,花了不少时间去看 apache tomcat japser的代码,并尝试替换 jasper-compiler-5.0.28.jar为Apache tomcat 6.0.18的最新版本,结果发现乱码问题的原因根本不在哪里。

appengine_web.xml里可以配置:<property name=”file.encoding” value=”UTF-8″ />。但是也不能解决乱码问题。

2009年04月11日

使用Google App Engine的URLFetchService实现类似phpproxy访问页面

Filed under: Java — 标签:, , — lizongbo @ 00:00

使用Google App Engine的URLFetchService实现类似phpproxy访问页面,
今天接着体验google appEngine,参考文档写了个类似phpproxy以代理方式获取页面内容的Servlet。
phpproxy的功能很强大,我的程序很简单,只实现了简单的获取页面内容然后显示。
phpproxy的代码在:http://idea.hosting.lv/a/phpproxy/phpproxy-0.6/phpproxy.phps
我的页面在:
http://lizongbo.appspot.com/p?url=http://www.google.cn
URLFetchService的介绍在:http://code.google.com/intl/zh-CN/appengine/docs/java/urlfetch/

首先,web.xml的配置是:

[code]
</servlet-mapping>
<servlet>
<servlet-name>HttpProxy</servlet-name>
<servlet-class>com.lizongbo.gapp.HttpProxyServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>HttpProxy</servlet-name>
<url-pattern>/p</url-pattern> <!– 写成/p/*居然没法匹配到/p–>
</servlet-mapping>
[/code]
com.lizongbo.gapp.HttpProxyServlet.java的代码为:

[code]
package com.lizongbo.gapp;

import java.io.IOException;
import java.io.PrintWriter;
import java.net.URL;
import java.util.Enumeration;
import java.util.Map;

import javax.servlet.http.*;

import com.google.appengine.api.urlfetch.*;

@SuppressWarnings(“serial”)
public class HttpProxyServlet extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws IOException {
String urlStr = request.getParameter(“url”);
java.net.URL url = null;
try {
url = new URL(urlStr);
if (!url.getHost().toLowerCase().endsWith(“appspot.com”)//不需代理本站
&& url.getPort() == url.getDefaultPort()) {//google限制了只能访问默认端口
URLFetchService urlFetchService = URLFetchServiceFactory
.getURLFetchService();
HTTPRequest req = new HTTPRequest(url, HTTPMethod.GET);
req.addHeader(new HTTPHeader(“X-Fetchby”,
http;//lizongbo.appsport.com“));
HTTPResponse res = urlFetchService.fetch(req);
response.setStatus(res.getResponseCode());
response.reset();
response.addHeader(“X-powerby”, “http://lizongbo.appsport.com“);
for (HTTPHeader header : res.getHeaders()) {
response.addHeader(header.getName(), header.getValue());
}
response.getOutputStream().write(res.getContent());
} else {
response.setContentType(“text/plain; charset=UTF-8”);
response.getWriter().write(
“url 不能是appspot.com,http请求端口必须是默认的80(http)和443(https)”);
}
} catch (Throwable ex) {
response.setContentType(“text/html; charset=UTF-8”);
ex.printStackTrace(new java.io.PrintWriter(response.getWriter()));
}

}
}

[/code]

在localhost方式访问某测试页面看到的请求信息为:
X-Fetchby=http;//lizongbo.appsport.com
User-Agent=Jakarta Commons-HttpClient/3.0.1
Host=3g.qq.com
部署到lizongbo.appspot.com之后访问看到的header信息:
X-Fetchby=http;//lizongbo.appsport.com
User-Agent=AppEngine-Google; (+http://code.google.com/appengine)
Referer=http://lizongbo.appspot.com/
Host=3g.qq.com
Accept-Encoding=gzip

上面的代码如果是在jsp里实现,
会因为Writer和ServletOutPutStream不能同时get而跑出异常,即使response.reset也没用。
异常信息如下:

[code]
java.lang.IllegalStateException: STREAM
at org.mortbay.jetty.Response.getWriter(Response.java:583)
at org.apache.jasper.runtime.JspWriterImpl.initOut(JspWriterImpl.java:122)
at org.apache.jasper.runtime.JspWriterImpl.flushBuffer(JspWriterImpl.java:115)
at org.apache.jasper.runtime.PageContextImpl.release(PageContextImpl.java:190)
at org.apache.jasper.runtime.JspFactoryImpl.internalReleasePageContext(JspFactoryImpl.java:115)
at org.apache.jasper.runtime.JspFactoryImpl.access$100(JspFactoryImpl.java:37)
at org.apache.jasper.runtime.JspFactoryImpl$PrivilegedReleasePageContext.run(JspFactoryImpl.java:173)
at com.google.apphosting.runtime.security.shared.intercept.java.security.AccessController_.doPrivileged(AccessController_.java:31)
at org.apache.jasper.runtime.JspFactoryImpl.releasePageContext(JspFactoryImpl.java:73)
at org.apache.jsp.proxy_jsp._jspService(p_jsp.java:82)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:237)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:63)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:125)
at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4547)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:4545)
at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)
at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:359)
at com.google.net.rpc.impl.Server$2.run(Server.java:792)
at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56)
at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:489)
at com.google.net.rpc.impl.Server.startRpc(Server.java:748)
at com.google.net.rpc.impl.Server.processRequest(Server.java:340)
at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:422)
at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319)
at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290)
at com.google.net.async.Connection.handleReadEvent(Connection.java:419)
at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:733)
at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207)
at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:101)
at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:249)
at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:373)
at java.lang.Thread.run(Unknown Source)

[/code]

Google App Engine的war文件夹里,我放了favicon.ico。但是不知为何却没到服务器上,导致后台老是看到访问favicon.ico的404错误。

另:google app engine说是支持Servlet 2.4,但是通过插件默认生成的web.xml里

dtd的引用是 http://java.sun.com/dtd/web-app_2_3.dtd

而web-app的属性声明里version的值却是2.5。

web Project引用的lib也是 /usr/local/java/appengine-java-sdk-1.2.0/lib/shared/geronimo-servlet_2.5_spec-1.2.jar。猜测应该是支持Servlet 2.5的。

2009年04月8日

Google App Engine Java runtime的file.encoding居然不是UTF-8

Filed under: Java — 标签:, , — lizongbo @ 23:45

Google App Engine Java runtime的file.encoding居然不是UTF-8
Google App Engine支持Java了,下午在Windows下测试Google App Engine Java Servlet的中文是正常的,但是测试的jsp页面显示却是乱码,
参考:http://618119.com/archives/2009/04/08/145.html

于是晚上写了下面的代码来取出系统属性和环境变量来看:
[code]

out.println(“系统属性有:《br/》”);

Enumeration e1 = System.getProperties().propertyNames();
String s1 = “”;
while (e1.hasMoreElements()) {
String key = (String) e1.nextElement();
String value = System.getProperty(key);
out.println(key + “=” + value + “《br/》”);

}
out.println(“系统环境变量有:《br/》”);
for (Map.Entry《String, String》 et : System.getenv().entrySet()) {
out.println(et.getKey() + “=” + et.getValue() + “《br/》”);

}
out.println(“你的请求属性有以下内容:《br/》”);
Enumeration e2 = req.getAttributeNames();
while (e2.hasMoreElements()) {
String key = (String) e2.nextElement();
Object value = req.getAttribute(key);
out.println(key + “=” + value + “《br/》”);

}
out.println(“你的请求头信息有以下内容:《br/》”);
Enumeration e = req.getHeaderNames();
String s = “”;
while (e.hasMoreElements()) {
String key = (String) e.nextElement();
String value = req.getHeader(key);
out.println(key + “=” + value + “《br/》”);

}

[/code]

程序运行结果如下:
系统属性有:
java.vendor=Sun Microsystems Inc.
java.specification.version=1.6
line.separator=
java.class.version=50.0
java.util.logging.config.file=WEB-INF/logging.properties
java.specification.name=Java Platform API Specification
java.vendor.url=http://java.sun.com/
java.vm.version=1.6.0_13
os.name=Linux
java.version=1.6.0_13
java.vm.specification.version=1.0
user.dir=/base/data/home/apps/lizongbo/2.332646483357654537
java.vm.specification.name=Java Virtual Machine Specification
java.specification.vendor=Sun Microsystems Inc.
java.vm.vendor=Sun Microsystems Inc.
file.separator=/
path.separator=:
java.vm.name=Java HotSpot(TM) Client VM
java.vm.specification.vendor=Sun Microsystems Inc.
file.encoding=ANSI_X3.4-1968
系统环境变量有:

你的请求属性有以下内容:
com.google.apphosting.runtime.jetty.APP_VERSION_REQUEST_ATTR=lizongbo/2.332646786822574367
你的请求头信息有以下内容:
Host=lizongbo.appspot.com
User-Agent=Mozilla/5.0 (X11; U; Linux x86_64; zh-CN; rv:1.9.0.8) Gecko/2009032712 Ubuntu/8.10 (intrepid) Firefox/3.0.8,gzip(gfe)
Accept=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language=zh-cn,zh;q=0.5
Accept-Charset=gb2312,utf-8;q=0.7,*;q=0.7

其中很奇特的是Google App Engine Java runtime的file.encoding=ANSI_X3.4-1968,而不是UTF-8。
从request的属性里看到了jetty字样,,不知google App Engine 的java运行环境是否和jetty有什么关系。
在Ubuntu下Eclipse里重新建立testweb,重新建立test.jsp,这次传上去再访问,中文显示就正常了。 测试地址:http://lizongbo.appspot.com/test.jsp

不明白是何原因。

Older Posts »

Powered by WordPress