Our first public test was only open to our IRC channel and twitter followers, conducted late at night (AEST) and was an obscure time to try and get people to participate. Within an hour we had hit our peak of concurrent connected players at 91 players on the server. There were many crashes involved with our streaming and enter / exit vehicle systems which needed to be addressed. At this time no vehicle theft system had been implemented and there were bugs associated with two players trying to drive the same vehicle.
With our second public test we decided we wanted to try and push for more players. Most of our fixes of the first test's issues had already been tested by our beta team, so we were confident with proceeding with another public beta test. This test introduced vehicle theft, a passenger system and we had introduced more vehicles into a confined area than ever before. This test showed us bugs with the client not properly announcing when a player had entered a vehicle as a driver, but had entered from the passenger side of the vehicle. This had some major sync issues (vehicles driving sideways) as well as some situations where multiple people would be trying to drive the same vehicle. The player peak at which we reached with this test was 251 concurrent players and beside the bugs, a lot of people appeared to enjoy the test. This test also showed us how unfriendly mods can be with our unfinished system. Players running some specific modifications on their game were able to crash entire sectors of players from our server. In order to run a test, we now had to stop work on making the game multiplayer and spend time to prevent mods from being loaded.
More! With the third public test many changes had been made with the client to make it more stable, the server had been tweaked for better performance utilizing data gathered from Test #2. We were very confident going into public test 3; we had announced ahead of time (by about 24 hours) that we would be running a large and extensive test over the proceeding weekend. We were amazed by the number of people joining when we brought the server up for testing. We were announcing the number of players live; within the 30 seconds we had over 300 players, crushing the record set by the previous test. The server peaked at having 650 concurrent players connected for this test, allowing us to collect over 10,000 bug reports of which about 100 were unique bugs. Of those 100 we narrowed it down to a few key parts of code that needed tweaking. The test was a huge success, it ran for some 72 hours, over the course of which the player count never dropped below 200 players (except for server restarts and new client updates). With this test, the popularity of JC2-MP grew; we had some 79,000 unique visitors to our website over the course of the test.
Exceeding our expectations immensely, we were blown away with how much of a success Public Test #4 had been. Approximately 20 minutes after bringing the server online, we had hit our soft player cap for that test at 1000 concurrent players connected to our server, showing us the level of stability that had been achieved with this build. With this test we decided to trial how the gameplay would be affected with the ability to steal vehicles from the stunt position of a car being disabled. This prevented players being able to use the grapple to quickly latch onto a vehicle (stunt position) and then quickly steal the vehicle from its current driver. The reactions were quite mixed; some people loved it while others hated it and cursed us for doing so. Nonetheless, it was an important gameplay decision to test. One thing that became quite clear with this, however, is just how powerful weaponized vehicles (particularly helicopters) became. This led us to concentrate on the implementation of on-foot weapons to help balance the situation out.
With Public Test #5 JC2-MP exhibited more stability than ever before - not only in terms of client stability, but in the gameplay itself. With this test we made major changes to how agents (other players in your game) operate, making them more like the local player as opposed to NPC's from the singleplayer. This paves the road for implementing additional synchronization within JC2-MP. Within 30 minutes of bringing the server up, we had once again hit 1000 players - the player count continued to climb to a further 1320 players, smashing our previous record for concurrent connected players. Over the course of the 24 hour test the server averaged 800 players online at a time, only beginning to dwindle as the test was coming to an end. After the conclusion of the test, it was determined that the server used 526GB of bandwidth for the 24 hour test and we had 10,505 players connect to our server.
Within 20 minutes of going live with the new build, we had crushed our previous record of 1,300 concurrent connected players. Within minutes after that, we had hit 1,800 players connected to our test server at the same time. With this we learnt of a limitation in our server code, which Trix quickly identified and rectified in a new version of the server and client. However, we lost some players and the server did not go above 1,600 players for the remainder of the weekend. Over the course of the weekend 22,420 unique players connected to our test server - these 22,420 players died 383,875 times. That’s 2.2~ deaths per second! 303,011 chat messages were sent and 313,418 teleports were taken!
Public Test #7 has been, by far, our worst beta-test yet, but it was still manageable. It had almost been a month since we had last ran a public beta-test for the community to take part in - this was mostly due to the core members of the JC2-MP development team being preoccupied with events unrelated to JC2-MP. We had made many alterations to both the server-side and client-side, allowing us to implement the server-side Lua scripting system. Over the first 24 hour period of the 48 hour test, Trix spent a lot of time profiling the server to pinpoint the cause of the performance drop on the server. Huge changes had been made to the server since the last test, so this was no small task. During this time 5-6 different server builds were tested, allowing for a process of rapid iteration. Every time, we could see the performance of the server improve moderately, until the core issue was discovered, isolated and fixed. This has opened our eyes as to the many improvements and optimizations that can be made to our server, and is most likely what we'll be focusing on to improve / fix for the next beta. In terms of statistics, there were 1,600 connected users at one moment during the weekend. We had 21,322 unique users join our server over the 48 hour testing period. These users died 223,704 times, averaging out at 1.29~ deaths per second! 167,030 chat messages were sent and 271,321 teleports were taken!
Only a few hours into this test, our beta server was attacked by a large DDOS attack. Even though the test was a lot shorter than we had hoped for, a sufficient amount of data was collected, allowing us to improve JC2-MP.