Map server spikes @ 99% CPU usage randomly
Warning! This is the old Hercules bugtracker archive, and may not reflect the current state of Hercules. The current bugtracker is on GitHub Issues.
Issue information
Issue ID
#5968
Status
Fixed
Severity
Medium
Started
Hercules Elf Bot
Jun 9, 2012 2:25
Last Post
Hercules Elf Bot
Jul 21, 2012 1:26
Confirmation
Hercules Elf Bot - Jun 9, 2012 2:25
Originally posted by [b]jTynne[/b]
Using trunk 16195 and Harmony, our map-server_sql keeps spiking randomly at 99% CPU usage even with all of my scripts disabled.
I've been monitoring my server for the past three days straight, modifying/updating all of my scripts in hopes that the issue was my fault. Even had the hosting company look into it and they came up with noticing that map-server_sql keeps spiking at 99% which effectively locks everything up for my players in-game.
This has been happening periodically all day long since I first started my server last Friday afternoon. We have ~40-60 players online average, however, the CPU spikes take place regardless of how many players are online.
If ANYONE can help, I will be forever grateful. Thank you!
Using trunk 16195 and Harmony, our map-server_sql keeps spiking randomly at 99% CPU usage even with all of my scripts disabled.
I've been monitoring my server for the past three days straight, modifying/updating all of my scripts in hopes that the issue was my fault. Even had the hosting company look into it and they came up with noticing that map-server_sql keeps spiking at 99% which effectively locks everything up for my players in-game.
This has been happening periodically all day long since I first started my server last Friday afternoon. We have ~40-60 players online average, however, the CPU spikes take place regardless of how many players are online.
If ANYONE can help, I will be forever grateful. Thank you!
Hercules Elf Bot - Jun 10, 2012 5:48
Originally posted by [b]jTynne[/b]
Disabling MVP Tombstones seems to resolve the lock-ups..
Disabling MVP Tombstones seems to resolve the lock-ups..
Hercules Elf Bot - Jun 11, 2012 4:44
Originally posted by [b]jTynne[/b]
[Error]: Memory manager: args of aFree 0x0xb27a3b24 is overflowed pointer db.c line 1053
[Warning]: ers::destroy : 11 entries missing (possible double free), continuing destruction (entry size=20).
[Warning]: ers::destroy : 1 entries missing (possible double free), continuing destruction (entry size=32).
[Status]: Close Map DB Connection....
[Status]: Close Log DB Connection....
[Status]: Finished.
[Error]: Memory manager: args of aFree 0x0xb27a3864 is overflowed pointer malloc.c line 621
[Error]: Memory manager: args of aFree 0x0xb27a3b24 is overflowed pointer db.c line 1053
[Warning]: ers::destroy : 11 entries missing (possible double free), continuing destruction (entry size=20).
[Warning]: ers::destroy : 1 entries missing (possible double free), continuing destruction (entry size=32).
[Status]: Close Map DB Connection....
[Status]: Close Log DB Connection....
[Status]: Finished.
[Error]: Memory manager: args of aFree 0x0xb27a3864 is overflowed pointer malloc.c line 621
Hercules Elf Bot - Jun 11, 2012 20:40
Originally posted by [b]Sharpienero[/b]
Do you have enough RAM?
Do you have enough RAM?
Hercules Elf Bot - Jun 11, 2012 20:49
Originally posted by [b]jTynne[/b]
Yeah, well over an extra gig above what's being used. =X
Yeah, well over an extra gig above what's being used. =X
Hercules Elf Bot - Jun 11, 2012 22:33
Originally posted by [b]Sharpienero[/b]
[quote name='jTynne' timestamp='1339447777' post='10407']
Yeah, well over an extra gig above what's being used. =X
[/quote]
Well from what I see, you might need extra ram.
Otherwise, I wouldn't know. Have you cleaned your system up at all with ccleaner or anything of the sort?
[quote name='jTynne' timestamp='1339447777' post='10407']
Yeah, well over an extra gig above what's being used. =X
[/quote]
Well from what I see, you might need extra ram.
Otherwise, I wouldn't know. Have you cleaned your system up at all with ccleaner or anything of the sort?
Hercules Elf Bot - Jun 11, 2012 23:26
Originally posted by [b]jTynne[/b]
System is optimized regularly; RAM usage doesn't spike. Only CPU usage. RAM is steady at around 20%.
Disabling MVP Tombstones fixed the spikes. Devs can start there.
System is optimized regularly; RAM usage doesn't spike. Only CPU usage. RAM is steady at around 20%.
Disabling MVP Tombstones fixed the spikes. Devs can start there.
Hercules Elf Bot - Jun 15, 2012 15:32
Originally posted by [b]wend[/b]
On MVP respawn while tombstone enabled CPU usage reached 100%.
On MVP respawn while tombstone enabled CPU usage reached 100%.
Hercules Elf Bot - Jun 15, 2012 22:39
Originally posted by [b]MarkZD[/b]
I confirm it.
This is a real error, you can reproduce it in a room full of MVPs, just put on some players in there with tomb system on and check your server lagging if not beeing completely locked.
Leave MVPs with a fast Respawn to check it ultra fast.
I confirm it.
This is a real error, you can reproduce it in a room full of MVPs, just put on some players in there with tomb system on and check your server lagging if not beeing completely locked.
Leave MVPs with a fast Respawn to check it ultra fast.
This post has been edited by MarkZD on Jun 15, 2012 22:45
Hercules Elf Bot - Jun 17, 2012 11:44
Originally posted by [b]wend[/b]
But i can't reproduce this on local server..
On live server with 150+ online it causes big lag
But i can't reproduce this on local server..
On live server with 150+ online it causes big lag
Hercules Elf Bot - Jun 17, 2012 14:12
Originally posted by [b]RiverStyx[/b]
Supposedly one of the memory leaks were found and fixed in April 24th, is it possible that fix was reversed or not fixed fully?
Perhaps someone can start [url=http://sourceforge.net/apps/trac/rathena/changeset/15954] here[/url]? Changeset 15954
Supposedly one of the memory leaks were found and fixed in April 24th, is it possible that fix was reversed or not fixed fully?
Perhaps someone can start [url=http://sourceforge.net/apps/trac/rathena/changeset/15954] here[/url]? Changeset 15954
This post has been edited by RiverStyx on Jun 17, 2012 14:14
Hercules Elf Bot - Jun 17, 2012 15:02
Originally posted by [b]jTynne[/b]
How has this not been tagged with a Severity of Critical yet?
How has this not been tagged with a Severity of Critical yet?
Hercules Elf Bot - Jun 17, 2012 18:20
Originally posted by [b]Ind[/b]
i wasn't able to reproduce this so far oo even with markzd extra info on how to reproduce
i wasn't able to reproduce this so far oo even with markzd extra info on how to reproduce
Hercules Elf Bot - Jun 17, 2012 18:27
Originally posted by [b]malufett[/b]
for all who confirm what processor speed do you used? AFAIK I core intel won't reached that high at any instance ([size=3]except booting up the server with low version of i core[/size]) ...XD
for all who confirm what processor speed do you used? AFAIK I core intel won't reached that high at any instance ([size=3]except booting up the server with low version of i core[/size]) ...XD
Hercules Elf Bot - Jun 17, 2012 18:58
Originally posted by [b]MarkZD[/b]
I tested it on a friend server.
It wasn't rA, it was Cronus Emulator(where Greenbox is the dev leader), so I believe the tomb is the same source or almost as it was implemented by him.
8gb shared ram, 1gb dedicated.
Intel Core2Quad Q9300 // Not sure if it's dedicated but server could handle more than 50 chars online(without tomb system). When tomb was on, it lagged during 5 sec, even if there were only 5 players on.
I said to this server owner turn off the tomb system and no more lag was noticed.
Linux CentOS 5
I tested it on a friend server.
It wasn't rA, it was Cronus Emulator(where Greenbox is the dev leader), so I believe the tomb is the same source or almost as it was implemented by him.
8gb shared ram, 1gb dedicated.
Intel Core2Quad Q9300 // Not sure if it's dedicated but server could handle more than 50 chars online(without tomb system). When tomb was on, it lagged during 5 sec, even if there were only 5 players on.
I said to this server owner turn off the tomb system and no more lag was noticed.
Linux CentOS 5
This post has been edited by MarkZD on Jun 17, 2012 19:01
Hercules Elf Bot - Jun 17, 2012 19:23
Originally posted by [b]Vali[/b]
Same problem here, when the tomb system is active and the MvP respawn the server suffer a lag spyke:
CPU: AMD Athlon 64 Dual Core (2x 1.8Ghz)
Memory RAM: 2 GB DDR2
OS: CentOS 5
Same problem here, when the tomb system is active and the MvP respawn the server suffer a lag spyke:
CPU: AMD Athlon 64 Dual Core (2x 1.8Ghz)
Memory RAM: 2 GB DDR2
OS: CentOS 5
Hercules Elf Bot - Jun 17, 2012 20:09
Originally posted by [b]wend[/b]
Local:
Windows 7, Intel Core 2 Duo E8400 - cant reproduce
VDS:
Debian 6, Xeon E5645 - lags on MVP spawn
Local:
Windows 7, Intel Core 2 Duo E8400 - cant reproduce
VDS:
Debian 6, Xeon E5645 - lags on MVP spawn
This post has been edited by wend on Jun 17, 2012 20:11
Hercules Elf Bot - Jun 17, 2012 20:16
Originally posted by [b]MarkZD[/b]
Apparently it can't be reproduced on windows, by looking the OS which lags is *nix.
Or not.
|
|
|
\/
Apparently it can't be reproduced on windows, by looking the OS which lags is *nix.
Or not.
|
|
|
\/
This post has been edited by MarkZD on Jun 18, 2012 19:35
Hercules Elf Bot - Jun 18, 2012 18:23
Originally posted by [b]benzanaz[/b]
^^
||
||
||
||
Yes I'm nix (CentOS)
confirm by me,
Intel Xeon/RAM 16GB/SSD 240GB + over player 1500above
Lag when boss respawn (after tome gone)
^^
||
||
||
||
Yes I'm nix (CentOS)
confirm by me,
Intel Xeon/RAM 16GB/SSD 240GB + over player 1500above
Lag when boss respawn (after tome gone)
This post has been edited by benzanaz on Jun 19, 2012 3:17
Hercules Elf Bot - Jun 18, 2012 21:52
Originally posted by [b]GreenBox[/b]
I've made a small optimization on tombstone system([rev=16318]), maybe it fix the cpu usage.
I've made a small optimization on tombstone system([rev=16318]), maybe it fix the cpu usage.
Hercules Elf Bot - Jun 19, 2012 13:31
Originally posted by [b]EvilPuncker[/b]
would be nice if anyone can test the change and give feedback
would be nice if anyone can test the change and give feedback
Hercules Elf Bot - Jun 19, 2012 19:00
Originally posted by [b]MarkZD[/b]
Try malufett's tomb system:
[url="http://rathena.org/board/topic/61486-src-mvp-tomb-ra-patch-for-added-features/"]http://rathena.org/b...added-features/[/url]
Try malufett's tomb system:
[url="http://rathena.org/board/topic/61486-src-mvp-tomb-ra-patch-for-added-features/"]http://rathena.org/b...added-features/[/url]
This post has been edited by MarkZD on Jun 19, 2012 19:31
Hercules Elf Bot - Jun 21, 2012 20:26
Originally posted by [b]benzanaz[/b]
still lag...
T-T
still lag...
T-T
Hercules Elf Bot - Jun 22, 2012 10:23
Originally posted by [b]benzanaz[/b]
@bumb
follow the update at r16318
the server is still LAG
not reslove.... need to fix again
@bumb
follow the update at r16318
the server is still LAG
not reslove.... need to fix again
Hercules Elf Bot - Jun 22, 2012 17:37
Originally posted by [b]GreenBox[/b]
That wasn't a fix, just an optimization
Sincerely I don't know what can cause this on mvp tomb unloading as it only unload the npc.
That wasn't a fix, just an optimization
Sincerely I don't know what can cause this on mvp tomb unloading as it only unload the npc.
Hercules Elf Bot - Jun 23, 2012 6:40
Originally posted by [b]benzanaz[/b]
@Greenbox
i'm still waiting you for fix this problem
now my server temp to turn off this option
@Greenbox
i'm still waiting you for fix this problem
now my server temp to turn off this option
Hercules Elf Bot - Jun 27, 2012 10:35
Originally posted by [b]benzanaz[/b]
bumb need to fix...
bumb need to fix...
Hercules Elf Bot - Jul 11, 2012 10:28
Originally posted by [b]spiritualkid[/b]
Anyone found a solution to this?
From my testings, this only occurs on UNIX (couldn't reproduce on Windows).
Also, the lag spikes were minimal when the player who previously killed the MVP was within the tomb's range (minimal as in barely noticeable), but the lag spikes are much worst (few seconds freeze) when the player isn't in the same map as the tomb.
Anyone found a solution to this?
From my testings, this only occurs on UNIX (couldn't reproduce on Windows).
Also, the lag spikes were minimal when the player who previously killed the MVP was within the tomb's range (minimal as in barely noticeable), but the lag spikes are much worst (few seconds freeze) when the player isn't in the same map as the tomb.
Hercules Elf Bot - Jul 11, 2012 14:10
Originally posted by [b]Ind[/b]
this makes no sense at all, the tombs are ordinary npcs with no special handling or anything like that at all.
[hr]
I've tried to reproduce by killing several mvps that spawn tombs -- doppelganger/baphomet/moon/gtb/maya/others and watched the cpu usage closely, it never gone beyond 1%
this makes no sense at all, the tombs are ordinary npcs with no special handling or anything like that at all.
[hr]
I've tried to reproduce by killing several mvps that spawn tombs -- doppelganger/baphomet/moon/gtb/maya/others and watched the cpu usage closely, it never gone beyond 1%
This post has been edited by Ind on Jul 11, 2012 14:40
Hercules Elf Bot - Jul 11, 2012 15:17
Originally posted by [b]jTynne[/b]
Perhaps in conjunction with something else besides "just" killing them. Unfortunately, this is still an issue for us.
Perhaps in conjunction with something else besides "just" killing them. Unfortunately, this is still an issue for us.
Hercules Elf Bot - Jul 12, 2012 17:13
Originally posted by [b]benzanaz[/b]
not test/ but i'm thinking about x64 compile (linux)
not test/ but i'm thinking about x64 compile (linux)
Hercules Elf Bot - Jul 17, 2012 2:33
Originally posted by [b]Raizen[/b]
in CentOS 6.2 and 5.8 x64 i cannot reproduce the spike.
in CentOS 6.2 and 5.8 x64 i cannot reproduce the spike.
Hercules Elf Bot - Jul 17, 2012 3:22
Originally posted by [b]Lighta[/b]
We had that spike on debian64 bit too, but it was before optimisation I'll try again, but it occur more on tomb deletion than spawning new one.
We had that spike on debian64 bit too, but it was before optimisation I'll try again, but it occur more on tomb deletion than spawning new one.
Hercules Elf Bot - Jul 21, 2012 1:26
Originally posted by [b]Ind[/b]
I identified the problem and believe I fixed it, check [rev=16460]
Before my fix: cpu spike at 36% when unloading tomb
after my fix: cpu spike at 0.6% when unloading tomb.
I identified the problem and believe I fixed it, check [rev=16460]
Before my fix: cpu spike at 36% when unloading tomb
after my fix: cpu spike at 0.6% when unloading tomb.