浅析goland等待锁问题
问题描述:
向后台发送了一个URL请求,竟然一直卡住,没有返回,就一直卡着
问题分析定位:
一开始还以为是网络,还是什么其它奇怪的原因,毕竟之前好好的。
这里应该第一时间思考环境的变化,网络、程序版本、还是什么地方变化了。
后来又试了好几次,发现还是一样,想到了将Pod的数目改成了2个,于是估计是等待锁或者死锁之类的。
通过以下链接获取调试信息:
curl“127.0.0.1:43411/debug/pprof/goroutine?debug=1”>1.out
curl“127.0.0.1:43411/debug/pprof/goroutine?debug=2”>1.log
搜索卡住的请求方法名,果真搜到一些,goroutine17002[semacquire,5minutes]:表示17002这个goroutine正在等待获取锁,等了5分钟了。
又有一个正在运行中的同名方法,估计是大家都在等它的锁,看来代码,果然有一个锁。
但是这个goroutine调用栈最上层是SetDataNodeCarry,看到其方法内部,发现其也加了一个锁,不过其就是简单的对一个字段的变更进行加锁。
分析其使用的锁相关的代码,大家没有重叠,每一次加锁只不过是保证数据的原子性和一致性而已,所以不会有死锁的问题。
阅读SetDataNodeCarry附近的代码,有一个foravailCarryCount 总结 遇到此类问题首先不能慌,之后不能逃避问题。 需要将问题尽可能地记录,以便于还原,同时还要基于现在的情况进行调试,千万不能想着重启会好,不能逃避问题。 后面解决问题的步骤还可以,不过这个问题本身也不是很难。 ------------记录 总结 到此这篇关于goland等待锁问题的文章就介绍到这了,更多相关goland等待锁内容请搜索毛票票以前的文章或继续浏览下面的相关文章希望大家以后多多支持毛票票! 声明:本文内容来源于网络,版权归原作者所有,内容由互联网用户自发贡献自行上传,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任。如果您发现有涉嫌版权的内容,欢迎发送邮件至:czq8825#qq.com(发邮件时,请将#更换为@)进行举报,并提供相关证据,一经查实,本站将立刻删除涉嫌侵权内容。
1@0x47fc420x80fb0d0x8112700x80b86e0x7ba7580x7e06860x7f24fa0x81d7120x6ff9b40x7018b60x702c880x6fe9710x469581
#0x47fc41sync.(*RWMutex).Unlock+0xb1/usr/local/go/src/sync/rwmutex.go:113
#0x80fb0c[go文件路径]m.(*Pod).SetDataNodeCarry+0x6c/go/src/[go文件路径]m/t.go:777
#0x81126f[go文件路径]m.pts.ptsn+0x5f/go/src/[go文件路径]m/t.go:1059
#0x80b86d[go文件路径]m.(*t).ctcpd+0x83d/go/src/[go文件路径]m/t.go:453
#0x7ba757[go文件路径]m.(*c).cDP+0x1e7/go/src/[go文件路径]m/c.go:558
#0x7e0685[go文件路径]m.(*m).cDP+0x375/go/src/[go文件路径]m/handle_admin.go:353
#0x7f24f9[go文件路径]m.(*m).ServeHTTP+0x1659/go/src/[go文件路径]m/http_server.go:188
#0x81d711[go文件路径]m.(*m).handlerWithInterceptor.func1+0x81/go/src/[go文件路径]m/http_server.go:160
#0x6ff9b3net/http.HandlerFunc.ServeHTTP+0x43/usr/local/go/src/net/http/server.go:1995
#0x7018b5net/http.(*ServeMux).ServeHTTP+0x1d5/usr/local/go/src/net/http/server.go:2375
#0x702c87net/http.serverHandler.ServeHTTP+0xa7/usr/local/go/src/net/http/server.go:2774
#0x6fe970net/http.(*conn).serve+0x850/usr/local/go/src/net/http/server.go:1878
2@0x43c20f0x44c6090x44c5df0x44c37d0x47ecb90x7ba6ad0x7e06860x7f24fa0x81d7120x6ff9b40x7018b60x702c880x6fe9710x469581
#0x44c37csync.runtime_SemacquireMutex+0x3c/usr/local/go/src/runtime/sema.go:71
#0x47ecb8sync.(*Mutex).Lock+0x108/usr/local/go/src/sync/mutex.go:134
#0x7ba6ac[go文件路径]m.(*c).cDP+0x13c/go/src/[go文件路径]m/c.go:554
#0x7e0685[go文件路径]m.(*m).cDP+0x375/go/src/[go文件路径]m/handle_admin.go:353
#0x7f24f9[go文件路径]m.(*m).ServeHTTP+0x1659/go/src/[go文件路径]m/http_server.go:188
#0x81d711[go文件路径]m.(*m).handlerWithInterceptor.func1+0x81/go/src/[go文件路径]m/http_server.go:160
#0x6ff9b3net/http.HandlerFunc.ServeHTTP+0x43/usr/local/go/src/net/http/server.go:1995
#0x7018b5net/http.(*ServeMux).ServeHTTP+0x1d5/usr/local/go/src/net/http/server.go:2375
#0x702c87net/http.serverHandler.ServeHTTP+0xa7/usr/local/go/src/net/http/server.go:2774
#0x6fe970net/http.(*conn).serve+0x850/usr/local/go/src/net/http/server.go:1878
goroutine13994[runnable]:
sync.(*Mutex).Lock(0xc002300468)
/usr/local/go/src/sync/mutex.go:72+0x2c9
sync.(*RWMutex).Lock(0xc002300468)
/usr/local/go/src/sync/rwmutex.go:93+0x2d
[go文件路径]m.(*Pod).SetDataNodeCarry(0xc0023003f0,0x4024000000000000)
/go/src/[go文件路径]m/t.go:774+0x36
[go文件路径]m.pts.ptsn(0xc003b17810,0x2,0x2,0x0,0x3)
/go/src/[go文件路径]m/t.go:1059+0x60
[go文件路径]m.(*t).ctcpd(0xc0000ef090,0x0,0x0,0x0,0x0,0x0,0x0,0x3,0x6,0xc002275958,...)
/go/src/[go文件路径]m/t.go:453+0x83e
[go文件路径]m.(*c).cDP(0xc00223c000,0xc00286e673,0xe,0xc00286e65f,0x6,0x0,0x0,0x0)
/go/src/[go文件路径]m/c.go:558+0x1e8
[go文件路径]m.(*m).cDP(0xc0009aca90,0xabfd80,0xc0031a1260,0xc00296be00)
/go/src/[go文件路径]m/handle_admin.go:353+0x376
[go文件路径]m.(*m).ServeHTTP(0xc0009aca90,0xabfd80,0xc0031a1260,0xc00296be00)
/go/src/[go文件路径]m/http_server.go:188+0x165a
[go文件路径]m.(*m).handlerWithInterceptor.func1(0xabfd80,0xc0031a1260,0xc00296be00)
/go/src/[go文件路径]m/http_server.go:160+0x82
net/http.HandlerFunc.ServeHTTP(0xc000990300,0xabfd80,0xc0031a1260,0xc00296be00)
/usr/local/go/src/net/http/server.go:1995+0x44
net/http.(*ServeMux).ServeHTTP(0x10a3520,0xabfd80,0xc0031a1260,0xc00296be00)
/usr/local/go/src/net/http/server.go:2375+0x1d6
net/http.serverHandler.ServeHTTP(0xc0000dad00,0xabfd80,0xc0031a1260,0xc00296be00)
/usr/local/go/src/net/http/server.go:2774+0xa8
net/http.(*conn).serve(0xc003b19860,0xac1d80,0xc0009a9e40)
/usr/local/go/src/net/http/server.go:1878+0x851
createdbynet/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2884+0x2f4
goroutine17002[semacquire,5minutes]:
sync.runtime_SemacquireMutex(0xc0026fc460,0x419400)
/usr/local/go/src/runtime/sema.go:71+0x3d
sync.(*Mutex).Lock(0xc0026fc45c)
/usr/local/go/src/sync/mutex.go:134+0x109
[go文件路径]m.(*c).cDP(0xc00223c000,0xc00002a083,0xe,0xc00002a06f,0x6,0x0,0x0,0x0)
/go/src/[go文件路径]m/c.go:554+0x13d
[go文件路径]m.(*m).cDP(0xc0009aca90,0xabfd80,0xc0031a0620,0xc002988900)
/go/src/[go文件路径]m/handle_admin.go:353+0x376
[go文件路径]m.(*m).ServeHTTP(0xc0009aca90,0xabfd80,0xc0031a0620,0xc002988900)
/go/src/[go文件路径]m/http_server.go:188+0x165a
[go文件路径]m.(*m).handlerWithInterceptor.func1(0xabfd80,0xc0031a0620,0xc002988900)
/go/src/[go文件路径]m/http_server.go:160+0x82
net/http.HandlerFunc.ServeHTTP(0xc000990300,0xabfd80,0xc0031a0620,0xc002988900)
/usr/local/go/src/net/http/server.go:1995+0x44
net/http.(*ServeMux).ServeHTTP(0x10a3520,0xabfd80,0xc0031a0620,0xc002988900)
/usr/local/go/src/net/http/server.go:2375+0x1d6
net/http.serverHandler.ServeHTTP(0xc0000dad00,0xabfd80,0xc0031a0620,0xc002988900)
/usr/local/go/src/net/http/server.go:2774+0xa8
net/http.(*conn).serve(0xc003b18640,0xac1d80,0xc00298e940)
/usr/local/go/src/net/http/server.go:1878+0x851
createdbynet/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2884+0x2f4
goroutine14532[semacquire,11minutes]:
sync.runtime_SemacquireMutex(0xc0026fc460,0x419400)
/usr/local/go/src/runtime/sema.go:71+0x3d
sync.(*Mutex).Lock(0xc0026fc45c)
/usr/local/go/src/sync/mutex.go:134+0x109
[go文件路径]m.(*c).cDP(0xc00223c000,0xc003f74303,0xe,0xc003f742ef,0x6,0x0,0x0,0x0)
/go/src/[go文件路径]m/c.go:554+0x13d
[go文件路径]m.(*m).cDP(0xc0009aca90,0xabfd80,0xc00454a620,0xc004544800)
/go/src/[go文件路径]m/handle_admin.go:353+0x376
[go文件路径]m.(*m).ServeHTTP(0xc0009aca90,0xabfd80,0xc00454a620,0xc004544800)
/go/src/[go文件路径]m/http_server.go:188+0x165a
[go文件路径]m.(*m).handlerWithInterceptor.func1(0xabfd80,0xc00454a620,0xc004544800)
/go/src/[go文件路径]m/http_server.go:160+0x82
net/http.HandlerFunc.ServeHTTP(0xc000990300,0xabfd80,0xc00454a620,0xc004544800)
/usr/local/go/src/net/http/server.go:1995+0x44
net/http.(*ServeMux).ServeHTTP(0x10a3520,0xabfd80,0xc00454a620,0xc004544800)
/usr/local/go/src/net/http/server.go:2375+0x1d6
net/http.serverHandler.ServeHTTP(0xc0000dad00,0xabfd80,0xc00454a620,0xc004544800)
/usr/local/go/src/net/http/server.go:2774+0xa8
net/http.(*conn).serve(0xc002e17540,0xac1d80,0xc002a74780)
/usr/local/go/src/net/http/server.go:1878+0x851
createdbynet/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2884+0x2f4