XXXX客户系统健康检查报告一、综述 (3)二、系统环境 (3)核心业务系统 (3)三、工作日志 (4)四、系统健康检查详细列表 (4)1.核心业务系统 (4)五、日常维护建议 (10)六、故障出现时，系统处理的步骤 (11)七、命令介绍 (11)一、综述应XXXX客户要求，倚博公司中间件技术专家针对客户的核心业务系统做了一次完整的系统健康检查。
整体上讲，公司的核心系统运行比较稳定，故障率和客户业务系统的MTBF (Mean Time Between Failure)都比较小。
但从系统维护的管理工作的规范和完善角度讲，还未建立完善中间件的维护，监控的方案。
现阶段，主要是利用维护人员的现场应急处理经验保证系统的运行正常，未合理的规划维护流程，未确认中间件的监控指标，未建立完善的监控，预警和应急处理方案。
本文档总结了这次现场工作的各项情况，并根据现场维护人员的要求，增加了系统维护方法的扼要说明。
二、系统环境核心业务系统客户系统名称：核心业务系统OS CPU：8OS Memory: 16 GBOS: Red Hat AS 4KENERL:2.6.9-42.ELsmpJBOSS Application Server 4.2.3JDK：Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_17-b04) Java HotSpot(TM) Server VM (build 1.5.0_17-b04, mixed mode)三、工作日志2008年6月5日上午9：30 系统健康检查四、系统健康检查详细列表1.核心业务系统操作系统配置检查系统配置：系统内核参数：JAVA虚拟机配置检查虚拟机安装路径：/opt/jdk1.5.0_17/JBOSS配置检查安装路径：/opt/jboss/jboss-4.2.3.GA运行期系统状态检查运行期虚拟机状态检查运行期JBOSS组件状态检查启动目录core文件的检查在JBOSS启动目录中发现“core”文件和“hs_err_pid*.log”的错误日志文件，说明系统出现过JVM core dump和异常退出故障，故障出现在一个虚拟机的解压缩操作，希望能通过进一步与开发商，维护人员沟通找出具体的故障原因，并提出解决方案。
日志检查日志文件中发现错误信息如下：1)启动JBoss时报如下错误：出现过一次22:53:52,907 ERROR [MainDeployer] Could not create deployment: 新建文件夹/jboss-4.2.2.GA/server/default/conf/jboss-service.xmlorg.jboss.deployment.DeploymentException: - nested throwable: (ng.reflect.UndeclaredThrowableException)at org.jboss.system.ServiceConfigurator.install(ServiceConfigurator.java:196)at org.jboss.system.ServiceController.install(ServiceController.java:226)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at ng.reflect.Method.invoke(Unknown Source)at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)at $Proxy4.install(Unknown Source)at org.jboss.deployment.SARDeployer.create(SARDeployer.java:249)at org.jboss.deployment.MainDeployer.create(MainDeployer.java:969)at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:818)at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:782)at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:766)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)at ng.reflect.Method.invoke(Unknown Source)at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)atorg.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationIntercepto r.java:142)at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)at $Proxy5.deploy(Unknown Source)at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:482)at org.jboss.system.server.ServerImpl.start(ServerImpl.java:362)at org.jboss.Main.boot(Main.java:200)at org.jboss.Main$1.run(Main.java:508)at ng.Thread.run(Unknown Source)Caused by: ng.reflect.UndeclaredThrowableExceptionat org.jboss.system.ServiceCreator.install(ServiceCreator.java:236)at org.jboss.system.ServiceConfigurator.internalInstall(ServiceConfigurator.java:451)at org.jboss.system.ServiceConfigurator.install(ServiceConfigurator.java:171)... 36 moreCaused by: ng.Error: ERROR: it turns out the root cause is not always a runtime exception! at org.jboss.mx.server.MBeanServerImpl.handleInstantiateExceptions(MBeanServerImpl.java:1295) at org.jboss.mx.server.MBeanServerImpl.instantiate(MBeanServerImpl.java:1237)at org.jboss.mx.server.MBeanServerImpl.instantiate(MBeanServerImpl.java:286)at org.jboss.mx.server.MBeanServerImpl.createMBean(MBeanServerImpl.java:344)at org.jboss.system.ServiceCreator.install(ServiceCreator.java:157)... 38 morejboss-service.x配置错误，通过修改具体错误配置，并解决了这个问题。
2)NoClassDefFoundError错误，当前的类装载器不能找到这个类文件，频率：出现一次ng.NoClassDefFoundError: Could not initialize class org.jboss.aspects.security.SecurityActions$ContextInfoActionsat org.jboss.aspects.security.SecurityActions.getContextException(SecurityActions.java:467)atorg.jboss.aspects.security.AuthenticationInterceptor.authenticate(AuthenticationInterceptor.java:13 1)at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:66)org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:11 0)at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106) at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:240)at org.jboss.ejb3.stateless.StatelessContainer.localInvoke(StatelessContainer.java:210)at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:84)at $Proxy147.getInitConfigAsInteger(Unknown Source)at com.police.alarm.AlarmAction.execute(AlarmAction.java:28)at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)at com.police.filter.SystemUserFilter.doFilter(SystemUserFilter.java:61)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)at com.police.filter.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:37)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:524)at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)atorg.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)atorg.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:5at .JIoEndpoint$Worker.run(JIoEndpoint.java:446)at ng.Thread.run(Thread.java:619)NoClassDefFoundError错误，通过找到类所在的文件，并通过正确配置或使其在正确的类路径中来解决此问题。