Jay SenSharma
Here is a Simple demonstration of Debugging Jdbc N/W Connectivity issues. Some very basic and common tools and simple Jdbc programs helps us a lot in debugging the JDBC issues.
You can also refer to the WebLogic JDBC related some very common tips:
Common Jdbc Issues-1).
If you get the following Error:
&BEA-149205& &Failed to initialize the application ‘TestDataSource’ due to error weblogic.application.ModuleException: .
weblogic.application.ModuleException:
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:289)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:93)
at weblogic.application.internal.flow.DeploymentCallbackFlow$1.next(DeploymentCallbackFlow.java:387)
mon.ResourceException: Could not create pool connection. The DBMS driver exception was: The Network Adapter could not establish the connection
- – - – - – or - – - – - -
mon.ResourceException: mon.ResourceException: Could not create pool connection. The DBMS driver exception was: Io exception: The Network Adapter
Degugging the root Cause of above Issue:
The Above Exception: java.io.IOException: The Network Adapter could not establish the connection
It simply suggest that the Database URL may not be correct in the DataSource XML file locatied inside “&DOMAIN_HOME&configjdbc” directoryor may be the TNS name OR some n/w issue between WLS BOX and the DB Box. Please try the following to Double Check it.
Step1). Add JDBC Driver also in the Classpath or Better run “. ./setWLSEnv.sh”
(NOTE: While running the above script please use two DOTs like mentioned above. The first DOT represents that set the Environment in the Current Shell and the second DOT (./) Slash represents that pick up the Script from the current Location. Both DOTs are separated by a single space. Once u run “. ./setWLSEnv.sh” after that try to echo the values of $CLASSPATH and $PATH to make sure that the env is set properly)
Step2). Use WLS
dbping utility to test the Database Network Connectivity from the WebLogic Server Box:
java -classpath /bea103/wl_server103/server/lib/weblogic.jar utils.dbping ORACLE_THIN &dbUserName& &dbPasswoes& &dbURL&
java -classpath /bea103/wl_server103/server/lib/weblogic.jar utils.dbping ORACLE_THIN scott tiger databaseHostName:h
Step 3). Try doing a telnet to connect to the Database Box on the Database Listen Port like following Just to make sure that the Database has started listening on the Mentioned Listen Port or Not …..:
databaseHostName
————————————————————————–
Common Jdbc Issues-2).
Many times we see that DataSource configured on WebLogic Server was running fine sometimes back and suddenly we start seeing the following Error:
Caused by: java.sql.SQLException: Io exception: Got minus one from a read call
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387)
at oracle.jdbc.driver.PhysicalConnection.&init&(PhysicalConnection.java:420)
Degugging the root Cause of above Issue:
The error “Caused by: java.sql.SQLException: Io exception: Got minus one from a read call” clearly indicates the root cause of this issue. “We get the following error “Got minus one from a read call” if the Database goes under Maintenance…..or Database is in inconsistent State (Due to any reason like Database Maintenance or Database unmounting…etc ).
WLS is asking a JDBC driver for a connection and getting that exception, which means the DBMS or network dropped the socket during the driver-DBMS handshake.
The above error ensures that there is Nothing Wrong from WebLogic Side …we need to contact the Database Administrator only.
Step1). Open the Server Log and check the timestamp of the first occurance of that Error in the Server log.
Step2). Confirm with the Database Administrator…. What was the activity happening on the Database side at that time (90% cases u will find that the Database might be under Maintenance that time)
——-Make Use of Appropriate Debug Flags:———
-Dweblogic.resourcepool.max_test_wait_secs=xx
Where xx is the amount of time, in seconds, WebLogic Server waits for connection test before considering the connection test failed. By default, a server instance is assigned a value of 10 seconds. This command line flag manages failures, such as a DBMS network failure, which can cause connection tests and applications to hang for extended periods of time (for example, 10 minutes). If the assigned time period expires, the server instance purges and disables the pool (closes all connections and blocks further reserve attempts) and re-enables the pool as soon as it is possible to reconnect.
You can use the following Debug Flags:
-Dweblogic.debug.DebugJDBCSQL=true
-Dweblogic.log.StdoutSeverity=Debug
Use the Following WLST Script to Enable the JDBC related Debug Flags
“ExampleDebugJdbc.py”
—————————————————————–
user=’weblogic’
password=’weblogic’
url=’t3://localhost:;
connect(user, password, url)
cd(‘Servers/TestServer/ServerDebug/TestServer’)
startEdit()
set(‘DebugJDBCSQL’,’true’)
activate()
—————————————————————–
Common Jdbc Issues-3).
If you see that the Database is going into DataSource is moving to SUSPENDED/Disabled State….The first of all check the database connectivity. If the dfatabase connectivity is OK and WLS is able to establish the Connectivity with the database Successfully then Please try to use the “weblogic.Admin” utility to Restore (Resume) the DataSource.
You can use “weblogic.Admin” utility to Enable and Disable the Pool (DataSource) Just to Confirm whether the dataSource is actually active or not:
Step1). run “. ./setWLSEnv.sh” first in the same Shell prompt….then do the following:
Step2). To suspend:
java weblogic.Admin -url t3://localhost:7001 -username weblogic -password weblogic SUSPEND_POOL YourDataSourceName
Step3). To re-enable:
java weblogic.Admin -url t3://localhost:7001 -username weblogic -password weblogic RESUME_POOL YourDataSourceName
To test whether you are getting any Exception or Error while doing this….If yes u are getting any Error or exception then It means there May be some Database connectivity issue…
If a datasource gets destroyed then even if u are having DataSource Configuration Entries in the “config.xml” and the “&DOMAIN_HOME&/config/jdbc/*.xml” files …still u wont be able to see that datasource in the Monitoring Tab in the AdminConsole. Because as soon as the datasource gets destroyed (Due to N/W disconnect or someother reason) the MBean Object gets destroyed.
Common Jdbc Issues-4).
&Warning& &JDBC& &BEA-001153& &Forcibly releasing inactive connection
“weblogic.jdbc.wrapper.PoolConnection_oracle_jdbc_driver_T4CConnection@3″ back into the connection pool “TestDataSource”, currently reserved by: java.lang.Exception
at mon.internal.ConnectionEnv.setup(ConnectionEnv.java:291)
mon.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:314)
mon.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:292)
at mon.internal.ConnectionPool.reserve(ConnectionPool.java:425)
at mon.internal.ConnectionPool.reserve(ConnectionPool.java:316)
If we see above kind of exceptions in our Server Logs then definately we need to look into the Application Code.
“That is either your application reserving and Hholding the jdbc connections (they aren’t lost means leaked), or maybe they are hanging waiting for the DBMS to respond.”
Usually we see this Warning When the Application code obtains a JDBC connection from the WLS datasource, then not using it and not closing it, Means Just Holding the Connection Reference, for longer than your datasource/pool is configured to allow (IdleConnectionTimeout). Make sure that u close all the JDBC related Objects in a proper sequence. like exactly in the following
Order…Always close the ResultSet, Statement and Connection objects insode the finally{} Block in the following order:
try {resultset.close();}
catch (Exception rse) {}
try {statement.close();}
catch (Exception sse) {}
try {connection.close();
catch (Exception cse) {}
Common Jdbc Issues-5).
Many times we get the following kind of error/exception while using a Connection object:
java.sql.SQLException: Closed Connection
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:133)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:199)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:263)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:271)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:445)
at oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:3620)
at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:3853)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1374)
at weblogic.jdbc.wrapper.PreparedStatement.execute(PreparedStatement.java:99)
In these cases we need to make sure that you enable the “Test Connection On Reserve” is disabled. for your Connection Pool.This is the best way the pool can ensure That connection pool provides us a good connection when an application ask for it.
Also provide a simple SQL query for connection testing in the DataSource configuration like ‘SELECT * FROM DUAL’
Common Jdbc Issues-6).
If you get the following Error:
java.sql.SQLException: No more data to read from socket
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1200)
at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1155)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:279)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:194)
at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:853)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1145)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1267)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1882)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1847)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:301)
This error occurs if an applications that uses a database connections from the DataSource/Pool and When the application checked out a connection that has been already “timed out” or has been “staled”, still the application uses it to connect to the database, this error occurs.
In this case eother we need to start the Oracle Database Server or the Application Server which is maintaining the Connection Pool (Reseting the DataSource is also a valid option) sothat the staled connection should be cleared out and a new connection should be establised automatically.
Jay SenSharma
If you enjoyed this post, please consider or
to have future articles delivered to your feed reader.
Leave a Reply
You must be
to post a comment.
Recent Posts
Copyright (C)
Middleware Magic. All rights reserved. |Created Cluster with two rabbitMQ nodes. Configuration is as below for
rabbit1 and rabbit2 nodes.
1& CachingConnectionFactory connectionFactory = new
CachingConnectionFactory();
connectionFactory.setAddresses(&rabbit1:5672,rabbit2:5672&);
2& node types
rabbit1 - disc node
rabbit2 - ram node
3& producer and consumer programs sits on rabbit2 node(ie& ram node)
4& Producer sample code -
String QueueName = &Queue.&;
for(int m=0; m&50000; m++){
// send message
System.out.println(this.rabbitTemplate.getConnectionFactory().getHost());
this.rabbitTemplate.convertAndSend(m);
/*Thread.sleep(100);*/
5& consumer code -
String QueueName = &Queue.&;
public void run() {
System.out.println(&Consumer running host : & +
this.connectionFactory.getHost());
SimpleMessageListenerContainer container = new
SimpleMessageListenerContainer();
container.setConnectionFactory(this.connectionFactory);
container.setQueueNames(this.queueName);
container.setMessageListener(new MessageListenerAdapter(new
TestMessageHandler(this.connectionFactory.getHost()), new
JsonMessageConverter()));
container.start();
TestMessageHandler class sample code-
public TestMessageHandler(String hostName){
System.out.println(&Host: & + hostName);
this.hostName = hostN
// Handle message
public void handleMessage(int message) {
System.out.println(&handleMessage Host: & + this.hostName);
System.out.println(&Int : & + message);
6& Each node executed below policy
cmd& rabbitmqctl set_policy ha-all &^Queue\.& &{&&ha-mode&&:&&all&&}&
7& Started producer and consumer simultaneously. Could see host name as
&rabbit1& then stopped &rabbit1& node with &rabbitmqctl stop_app& command
to test fail-over scenario. Then got the below error
[.listener.SimpleMessageListenerContainer]: Consumer raised
exception, processing can restart if the connection factory supports it
com.rabbitmq.client.ShutdownSignalException:
reason: {#method&connection.close&(reply-code=541,
reply-text=INTERNAL_ERROR, class-id=0, method-id=0), null, &&}
com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:678)
com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:668)
com.rabbitmq.client.impl.AMQConnection.handleConnectionClose(AMQConnection.java:624)
com.rabbitmq.client.impl.AMQConnection.processControlCommand(AMQConnection.java:598)
com.rabbitmq.client.impl.AMQConnection$1.processAsync(AMQConnection.java:96)
com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:144)
com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:91)
com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:523)
[.listener.SimpleMessageListenerContainer]: Restarting
Consumer: tag=[amq.ctag-5CJ3YJYfMZDnJOnXsds6_Q], channel=Cached Rabbit
Channel: AMQChannel(),
acknowledgeMode=AUTO local queue size=0
after this warning, again am getting host name as &rabbit1& only.
actually it should be &rabbit2& as per my understanding but its not
happening.
So, here are my Queries -
1& Why am getting host name as &rabbit1& even after stopping?
2& To test the fail-over do we require any load balancer?
3& If my steps are wrong for testing fail-over case, please
provide steps for the same?
4& How to distribute queues/messages to particular node, as
below, 1-500 messages/queues to node1, 501-1000 messages/queues to node2,
5& Please let me know is there any other approach to test
fail-over scenario?
Appreciate any help on this.
-------------- next part --------------
An HTML attachment was scrubbed...
Previous message:
Next message:
Messages sorted by:Netty4.x分析 - wjp1991 - 博客园
官网定义：&netty是一个异步、事件驱动的网络应用框架，用于快速开发可维护的、高性能的服务端和客户端程序。
Architecture Overview
网络模型：netty采用了Reactor设计模式，Reactor设计可分三种：
单线程版本，如图：
学C的朋友会知道IO多路复用，我感觉和这个Reactor模式差不多，Reactor收到新连接调用acceptor的accept，返回的SocketChannel会注册到Reactor里，当连接可读或者可写时，分发一个handler处理。
多线程版本，如图：
处理部分增加了线程池。
Multi-Reactor版本：
监听端口注册到mainReactor里，有连接，调用accept，返回的连接注册到subReactor里，subReactor只负责读写，处理部分交给线程池。
Netty采用的方式类似于第三种，Netty3.6里mainReactor对应Boss类，subReactord对应NioWorker类；4.x里是实现EventLoopGroup接口的某个类，如NioEventLoopGroup（multithreaded event loop that handles I/O operation），EventLoopGroup相当于管理EventLoop的线程池，thread数量是可以配置的，echoServer例子中：
42,43行就是boss和worker了，ServerBootstrap是设置服务器的帮助类。
47行用NioServerSocketChannel类说明后面会用它去实例channel来接受incoming连接。
48行option方法可以指定Channel实现的方式。
50行：subReactor监听的channel来事件了，处理方法要通过childHandler方法指定，这是需要我们实现的，childHandler方法的参数是ChannelHandler接口的某个类，然后回调；拿FactorialServerInitializer举例，层次关系如图：
ChannelInitializer用来配置channel，这里要实现抽象类ChannelInitializer里的initChannel方法，意味着要在initChannel方法里配置pipeline。
我们观察EventLoopGroup类，如下图所示：
EventExecutor和EventLoopGroup都包含通用的&event loop API；EventLoopGroup有register方法，提供向其注册channel，返回ChannelFuture；
Netty Pipeline：
每个channel都有自己的pipeline，channel创建则对应的pipeline自动创建，下图显示了IO事件如何通过ChannelHandler在ChannelPipeline中处理的：
在pipeline里，每个stage运行一个InboundHandler或OutboundHandler，设计过MIPS经典五段pipeline的朋友应该知道锁存器设计，这里对应ChannelHandlerContext，ChannelHandlerContext可以通知ChannelPipeline里下一个ChannelHandler工作，并把事件流传给下一个ChannelHandler，也可以动态修改它所属的ChannelPipeline；
Inbound事件流传递方法：
Outbound事件流传递方法：
&我们看下官网的例子，io.netty.example.factorial&这个包，Pipeline部分（服务端）：
当Socket.read（）发生时，handler处理事件的顺序是：BigIntegerDecoder-&FactorialServeHandler
当Socket.write（）发生时，handler处理事件的顺序是：NumberEncoder
ChannelInboundHandlerAdapter的方法：
ChannelOutboundHandlerAdapter的方法：
关于Pipeline的一些说明：并不是一个阶段执行完了，才去执行下一个阶段，而是每个Handler有对应的事件处理方法（如上ChannelInboundHandlerAdapter和ChannelOutboundHandlerAdapter中的方法），当这个Handler接收到了某事件，就会调用这个事件处理方法，然后会触发下个Handler对应的事件处理方法，下面用自带的io.netty.example.http.helloworld包验证这个想法（在方法里插了输出）：
Socket.read（）：HttpServerCodec-&HttpHelloWorldServerH
Socket.write（）：HttpServerCodec
Zero Copy：
Netty里用到了ZC技术，是介绍ZeroCopy比较好的一篇文章，Netty里FileRegion就是用来支持ZeroCopy的接口，ZC在传输大文件时比较有优势，把大文件指定到Channel上，直接传输，不经过Application层。
&Channel包：
Channel接口封装了socket，提供IO操作（如read，write，connect，bind等）的组建，具体属性、方法可参考文档；
ChannelFuture接口继承了io.netty.util.concurrent.Future，Netty中IO调用均是异步的，调用立即返回，返回结果记录在ChannelFuture里，ChannelFuture随着IO操作的开始而被创建，它的状态可以是完成或者未完成，初始态是未完成，状态如下：
我们可以通过向ChannelFuture里增加和删除ChannelFutureListener（继承GenericFutureListener）（通过addListener（s）、removeListener（s）方法），IO操作完成时触发GenericFutureListener的operationComplete方法执行，这是异步的操作；我们也可以调用ChannelFuture的await方法阻塞Control Flow，直到ChannelFuture完成。
ChannelPipeline已在上文介绍；
Buffer包：
Netty使用自己的buffer API处理字节序列，而不是使用NIO自带的buffer，这样的定制有很多优势吧，官方文档是这么说的：在常见的网络应用中，我们会有一些buffer，它们经常需要组装成一个buffer，netty提供composite buffer，它允许你把已经存在的几个buffer组合起来创建一个virtual buffer，不需要内存拷贝：
还有许多协议的MTU都是不确定的，Netty允许你创建动态大小的buffer，来降低内存开销。
Channel读的数据会写到实现ByteBuf接口的某个类里，ByteBuf里数据满了，会调用handler处理，io.netty.handler.codec包里面会有一些类把package frame，也就是收到的ByteBuf decode成Message，交给handler处理；ByteBuf提供Java nio缓存（ByteBuffer）类似的方法，ByteBuf接口的实现层次图：
buffer包中有一个帮助类Unpooled，用于创建ByteBuf，所有的ByteBuf都是通过ByteBufAllocatore和UnpooledByteBufAllocator分配的，在Unpooled类里，默认的分配器是UnpooledByteBufAllocator，默认分配的ByteBuf类型是UnpooledHeapByteBuf；
HeapByteBuf是在Java堆上分配内存；DirectByteBuf用的是NIO的ByteBuffer，CompositeByteBuf是一个虚拟的buffer，将多种buffer合并成一个buffer；
细心的同学会看到PoolByteBuf和UnpooledByteBuf，PoolByteBuf是4.0新引入的，设计思想借鉴jemalloc（core：混合了slab分配器和buddy分配器），优点是减少内存碎片，Slab分配器是基于对象管理的，分配对象，直接从Slab系统里拿，无需再次初始化，释放对象，则保留在Slab系统里，标记为脏，不需释放，降低GC压力；
&io.netty.handler.codec包：
ByteToMessageCodec类封装了ByteToMessageDecoder和MessageToByteEncoder（都作为Pipeline Handler）；我们先观察ByteToMessageDecoder，其可以将流式的字节转化成消息类型，他有一个成员cumulation（ByteBuf类型），收消息时会把收到的msg（ByteBuf类型）传递给cumulation，数据准备好后调用callDecode，callDecode进一步调用decode方法（具体分帧方法），这个方法交给子类实现；MessageToByteEncoder里的write方法同理，write里调用子类实现的encode方法将消息encode成ByteBuf，发送出去。
Netty in Twitter：
写Netty的同学在twitter工作，twitter的新搜索架构Blender就是基于Netty的，如果您能FQ，可以访问，否则中文在，英文在，来了解Blender的架构。