Posted this evening on the America’s Army Official Forum (by AACM Ed “Phoenix” DeGorter on the Colonel’s behalf):

Since the launch of America’s Army 3 (AA3) on June 17, America’s Army has seen data transfer demands upon our game infrastructure that far exceeded our expectations and capacity to support new game registrations and game play. Data transactions associated with all existing and new players attempting to complete the Basic Combat Training (BCT) portion of the AA3 game over the span of a few days overwhelmed our authentication and database systems. These sections of AA3 include key new features of the game that are particularly data intensive and which we clearly did not adequately load test during our closed and open beta process. As Director of the Army Game Project, I apologize for this gap in service and want to outline below the actions we have taken and are taking to close this gap.

First, AA3 is designed to provide players a great deal of data regarding their performance and key aspects of the online experience. As a result, compared with prior versions of America’s Army, AA3 generates about ten times more traffic between game instances and our middleware and backend infrastructure. While we accounted for this level of data exchange when designing our AA3 infrastructure, we did not anticipate the unprecedented demand and data transfer levels we are witnessing as tens of thousands of players are simultaneously establishing accounts and working through the BCT portion of the Game. This was due to the fact that we did not adequately replicate the data transaction loads associated with this game activity during our beta test. As a result, immediately following the launch of AA3, we saw a mismatch between incoming data transactions and our infrastructure capacity. Therefore, we immediately began adjusting our infrastructure resources to maximize capacity and undertook substantial tuning of our database systems. Shortly after launch, we also discovered a hardware fault at our data center that created significant network lag. After addressing these faults, we achieved improvement in our infrastructure performance. However, demand for the game quickly consumed the capacity we created with the foregoing steps. Therefore, while we were able to support additional users, individual players did not see substantial improvement in game performance.

Since last Friday, to further improve system performance, we created a data handling modification for the game designed to moderate data flows during this launch period. This modification was released on Friday evening via Steam and our Deploy Client to automatically upgrade AA3 games installed on players’ computers. Additionally, we began crediting all players for completion of BCT levels so that they could proceed directly to multi-player missions; thereby removing the bottleneck created by the high volume of players engaging in BCT maps and improving the overall player experience.

We also began addressing performance and stability issues on the game servers by implementing memory upgrades and adding more database nodes to carry the load of authentication and player stats traffic between game instances and our infrastructure. We are correcting map rotation reporting deficiencies and issues with the Master Browser System (MBS) that prevent players from identifying servers with available Soldier positions. To improve network traffic, we continue to tune game network code, add more game servers, and correct server administration features so that server providers can better manage their servers. Finally, we are remedying instances where we identified memory leaks and are reducing server memory requirements to eliminate network lag while increasing server stability.

In summary, we are working to add capacity, and correct faults that have significantly impaired player access to America’s Army 3. We appreciate player patience as we ramp up our infrastructure to meet the demand that clearly exceeded our capacity on launch day.

Casey Wardynski
Colonel, U.S. Army
Director, Army Game Project (America’s Army)

If you haven’t done so already, head on over to the America’s Army Forums,  check it out, and join the conversation!

In a post on the America’s Army Forums, Community Manager Ed “Phoenix” DeGorter made the following statement:

Attention Soldiers!

Now available via Steam and the Deploy Client is a patch to greatly improve performance of AA3. This patch addresses the following issues:

• Removed apparent lag caused by other players entering or leaving a server.
• Resolved many start of round spawn issues.
• Resolved problem with training results not saving.
• Improved communication with Authentication server.
• Resolved freezing up when moving from one training mission to another.
• Cleared up hesitation during training missions
• Significantly reduce number of authentication transactions between player and Authentication server.
• Resolved achievements save and display problems.
• Improved login communication with Authentication server
• Improved player statistics tracking for better accuracy
• Improved overall performance.

Get the patch now! See you on the battlefield.

As mentioned in the statement, the patch is now available via both Steam and the Army’s Deploy Client. In order to get the update on Steam, make sure that you close out Steam entirely (right click the tasktray icon, and select “exit”) and then restart Steam from the Desktop or Start menu. The client should immediately begin downloading the patch, which appears to be fairly small (took about 5 min to download here).

So, get the patch, and get those boots on the ground. This weekend should be epic!

If you’re one of the happy few who are having this problem, don’t worry. Read on to find out how to fix.

I had the same problem too, and upon doing some research, found that this problem will resolve if you close out the Steam client entirely, then restart it. This means making sure that you right click the Steam icon in the Task Tray (down by the clock), and select “Exit”. Then open Steam again form the desktop or start menu.

If at first this dosen’t work, you might need to repeat the process a couple of times to whip your Steam client into compliance. Once it does kick in, you can expect to wait a few minutes while the America’s Army files are decrypted, then you should be golden.

Mine just finished here, so I’m off to play (barring no other unforseen errors).

See you on the battlefield!

–
-=XR=-
Site Admin

Users who preloaded on Steam reported problems with getting their Steam client to recognize the release and unlock their game (myself included). If you did preload AA3 on Steam, you need to close out Steam entirely and start it back up, maybe several times, for the client to begin decrypting.

While the America’s Army website was overwhelmed earlier,  the Army issued the following statement on the AA3 Facebook page:

None of us could have imagined the incredible number of players who would try to download and install this game at the same time. Due to this massive number of simultaneous connections, we’re having some trouble keeping up with the demand. You should see an increase in both accessibility and gameplay performance very soon. Please see the full post on the Discussion Board for more info.

The Discussion Board post (also on the AA3 Facebook page):

As we counted down to this release at 3PM today, none of us could have imagined the incredible number of players who would try to download and install this game at the same time. Due to this massive number of simultaneous connections, we’re having some trouble keeping up with the demand.

This is also placing a tremendous load on our Authentication Servers, which may temporarily be causing some gameplay performance issues.

That’s the bad news.

The good news is that absolutely everyone here at the Army Game Project, and our Partners, are working to resolve this problem, and you should see an increase in both accessibility and gameplay performance very soon. Once the load balances out a bit, both these elements will improve dramatically.

We are grateful for the overwhelming interest shown for America’s Army 3, and we appreciate your patience. All issues will be resolved as soon as possible.

If you have issues or concerns, please email our Community Managers at aacm@americasarmy.com.

Thanks,

America’s Army Game Project

My Steam client finally just finished decrypting. Time to see what all the hubbub is about.

See you on the battlefield!

–
-=XR=-
Site Admin

For more information visit http://store.steampowered.com/app/13140/

Just bear with us, we’ll get it under control. Please stop back!

Thanks,

The bad news, is that FilePlanet only has a limited number of beta keys available to hand out, so you need to get in on it ASAP. The good news is, it won’t cost you a dime! Keys are available to paying and non-paying Fileplanet subscribers alike!

So what are you waiting for? Move on out and be the first to play America’s Army 3 before anyone else! Contribute to the process of making this yet another successful release!

To get started, head on over to FilePlanet now!

However, the crew of OEF is still alive, and we’re ready to bring OEF Gaming back to its former glory, as one of the top America’s Army gaming communities on the web.

Check back soon for more information.

If you were or are interested in becoming a member of the OEF team, drop us a line!

Thanks,

XR
Owner & Site Admin
OEF Gaming