THE IMPORTANCE OF TESTING.

Posts

Pages: first 12 next last
An interesting topic popped up in my email about the different levels of testing.
Playtesting is Sovereign pt. 1

Here's an older article on how far Bungie went in testing Halo 3:
Halo 3: How Microsoft Labs Invented a New Science of Play

As one of the vital steps of professional game design, and one that involves interaction with others, I'm very curious as to how others have tackled this. Recruitment, instruction, reporting, followup, feedback, retesting, communication methods (email, im, irc), scheduling, compensation, how have any of these factored into your process?

I'm sure many here have come up with their own system, what have you done that works? What didn't work, and why? What would you do that's different?

halibabica
RMN's Official Reviewmonger
14893
Back in the day, when I actually completed stuff in a timely fashion, I would first run a complete playthrough myself. Then I had my dad and sister play through while I watched and took notes on stuff to fix. Then I would run another test myself just to make sure, and post it after that.

Nowadays I can't count on my family anymore, so I'm trying to get outside help. So far...I dunno. I'm still waiting on my tester. It's not particularly organized, but I think it'll turn out okay. Once it turns out at all, of course.
Max McGee
with sorrow down past the fence
9219
After a certain point, playing your own games becomes a vile, disgusting, deleterious task. This is entirely unrelated to the quality of your game, and has everything to do with the nature of repetition. After all, familiarity breeds STFU. No matter how good your game is, if you are playtesting it *enough* you will *HATE* it.

In terms of the actual mechanics behind the social engineering involved in recruiting and coordinating beta testers, it's a very interesting topic, but it seems a bit esoteric and advanced to be discussed at the amateur level. It is hard enough to round up ten people and get even two of those ten who agree to beta-test to play the full game even once all the way through. I don't think having a sophisticated schema will really do anything to improve that basic fact. After all, we're not Blizzard or Bungie. Not even close. There's no doubt about who's doing who a favor in the playtester/developer relationship.
Craze
i bet she's a diva with a potion popping problem
13865
post=154450
There's no doubt about who's doing who a favor in the playtester/developer relationship.


Tell that to the Alter AILA: Genesis testers!
Max McGee
with sorrow down past the fence
9219
I feel pretty meh about that game, I dunno.
Craze
i bet she's a diva with a potion popping problem
13865
Good thing I wasn't talking about you.
LockeZ
I'd really like to get rid of LockeZ. His play style is way too unpredictable. He's always like this too. If he ran a country, he'd just kill and imprison people at random until crime stopped.
6003
Basically I just run through the stuff myself. I keep about 40 save files - named and numbered - at almost every major spot you could save during the game.

Sometimes I still miss things. But this isn't very common when making changes to an existing game. If I'm adding a specific new thing to the game, I generally know exactly what I'm adding and how to test it.

When I was first making the game, it was a lot more chaotic. There was no way I could find all the problems myself. That's the situation where you really need extra testers, I think. Honestly, you know what I did? I gave the game to friends - not as testers, but just telling them that it was finished. And told them I didn't think there were any bugs, but if they found any, to please let me know. You'd be amazed how well that works. In a couple cases I even got to stand behind them and watch as they played parts of it, which was insanely helpful. The only downside is, if you were hoping your best friends would enjoy the game, they probably won't as much, because it will, uh, have some bugs.
post=154455
I feel pretty meh about that game, I dunno.

Yeah, well, I can officially say it is awesome. So... like... shut up?


In the past I have had my close friends, the ones who like RPGs, do play-testing for me (after having played the game myself). After they are done, I'd test a couple more times myself and then release. However, as this has seen many bugs slip through the cracks in the past, I am considering recruiting beta-testers who are also developers as I think they are more likely to spot bugs since they are likely to have experience testing their own games...
Let's see, currently most of the time I'm spending is programming fixes and additional stuff for a multiplayer server, and at the moment my testing scheme basically goes.

Compiles Server->Does it start?->Can you log onto the server from whatever's running it?->Can you log in over LAN?->Does what I've added or fixed work?->Replaces current online server->Crashes an hour later and then four to five hours later server guy sends me error log and restarts server

I really need to set myself up a proper test server on a VPS or something. :/
Despite
When the going gets tough, go fuck yourself.
1340
I just find myself drunk or baked one quaint evening with nothing to do so I play through what I got so far and see if it works.

Rinse, repeat.
After a certain point, playing your own games becomes a vile, disgusting, deleterious task. This is entirely unrelated to the quality of your game, and has everything to do with the nature of repetition. After all, familiarity breeds STFU. No matter how good your game is, if you are playtesting it *enough* you will *HATE* it.


Wrong. I actually call this the interest regression test, if the novelty of the game has worn off yet you don't mind the task of play testing and trying to break your own stuff, you're doing good. Especially if you're also writing the engine and everything, I think my replay count has gone into the hundreds/thousands. Mind you I also have to press every little button, use every item, switch around every party member, and so much more. If I feel as excited as I do programming the thing, I'm doing good.

I love the stream of feedback I got, but all of a sudden it went dead right when I had something deliverable! I've got the link to the 0.1a bug test thingy and the chatter just went dead. I've never had this happen before, really.

The article kind of overrates playtesting, but it is still vitally important to have done. When I've got people right there beside me wanting nothing more than to trash up all my deliverables, that's when I'm having the most fun. Even responses to just screenshots lets me know what people are thinking.


Also as a side note, feel free to PM me if you want me to thrash up your game test demos.
I've been working on my game for a long time now I'm starting to get bored of all my test runs.
I'm glad it's almost done. However, my concern is the difficulty level. I am developing an action-RPG, which can have a varying difficulty level depending on how experienced one would be with the game.

Since I'm the developer I kinda know EVERYTHING about it already. The enemy's movement patterns, the bosses tactics, etc. I find the level just fine - for me that is, but I've noticed a few occasions when letting friends try it out that it can be a quite frustrating.

And when even I myself have problems in some situations - uh-oh...
Puddor
if squallbutts was a misao category i'd win every damn year
4979
I like to think that if you can find the tedious parts entertaining still you're good. I can run through my game and grind to hell and back, the battles are exactly how I wanted them to be and I really just enjoy playing. Only problem I have with starting new saves and testing is PLAYING THROUGH MY STORY AGAIN. I'm very prone to bugs and I keep a playthrough save so I'll see most cutscenes three or four times at least. I can quote a few of them word by word, namely the intro scene. I have rewritten most cutscenes a few times now too just to keep them interesting.

I'm actually thinking about making a mid-way point release just to make sure the game's clean up to...well mid-way. But I'm nervous about doing it, heh :/


Only problem I have with starting new saves and testing is PLAYING THROUGH MY STORY AGAIN


I bet you if more people would just playtest their own games, we would have less ... ... ... and
Max McGee
with sorrow down past the fence
9219
Wrong. I actually call this the interest regression test, if the novelty of the game has worn off yet you don't mind the task of play testing and trying to break your own stuff, you're doing good. Especially if you're also writing the engine and everything, I think my replay count has gone into the hundreds/thousands. Mind you I also have to press every little button, use every item, switch around every party member, and so much more. If I feel as excited as I do programming the thing, I'm doing good.


I think once you reach the hundreds of thousands of hours mark, which I've been at for years, that's no longer even possible.
You don't NEED to test your game. You can just do the Action 52 approach.
I'm reaching that stage where I have very little left to do in my game, but now I am edging my attention on a new game to make, while there is still small bit stuff to finish, I hate it when this happens.

I swear its hard to devote an entire day to do the last few things, which can easily be done in maybe an hours time, but as one person mentioned, the testing phase gets redundant after half a million play throughs.
Testing your own game can only fix so much. You will obviously see visual errors, spelling mistakes, an event not doing what it should.. but I've found that all my testing doesn't really do anything for the actual gameplay. I know the order of all the puzzles and how everything works so I there is no trial and error in solving it. There is no way to tell if the puzzle is challenging or if the solution will even reveal itself based on what is given to the player. That's the biggest one for me, I will make a puzzle but not give any real clues on how its solved but it's something I have started to take into consideration.

I agree that a developer would be better and finding bugs. Especially someone who knows the engine it was made with.

As far as testing your own game, I think it depends on the game. For most of you, your game is an RPG, so testing involves going through the text and maybe fighting monsters to level up even though your just trying to get to a certain area by playing the game as the player would. Those kinds of things might get boring after a while. I find that with my game, being an action game with no text as of yet, it is far more enjoyable to test things out.

Most of all I think that when you are making a game there are ways to organize things that reduce the amount of time needed to test, reduce the time it takes to fix problems, and just reduce bugs in general. The most important is creating test areas and working on the coding before you make all the large maps and place everything in it. This idea slapped me in the face(several times) as I was making changes to 100's of events several times over after making a slight update to one of the systems. Instead everything is now tested in 1 small room and only makes it out into the world map after it is known to work. Also, even though something is complete it may stop working when you program some other feature so it makes things much easier if you wait until everything is programmed and known to work together before you even start laying down the maps,plot, progression, towns, NPC's, quests, and so on.

I plan on writing an article describing some of the things I came to learn about organization which has made programming my game much easier.

Also, since I have lost a bit of interest in working on my game I would like to offer myself as a tester to anybody who asks. Just PM me. I may not be able to(or want to) play through your entire game/demo but I will play for at least an hour and provide feedback immediately. It is not expected, but would be appreciated that you test my demo in return :) It's not very long.
post=154651
Most of all I think that when you are making a game there are ways to organize things that reduce the amount of time needed to test, reduce the time it takes to fix problems, and just reduce bugs in general. The most important is creating test areas and working on the coding before you make all the large maps and place everything in it. This idea slapped me in the face(several times) as I was making changes to 100's of events several times over after making a slight update to one of the systems. Instead everything is now tested in 1 small room and only makes it out into the world map after it is known to work.

I am too familiar with this scenario. I'm also working on an action RPG in rm2k3.
I started out with a single map where I try out all the more complicated stuff.
However, there has been some later changes which still has forced me more or less to replace or edit a certain event that exists on almost all of my 400+ maps. This has occurred several times...
kentona
I am tired of Earth. These people. I am tired of being caught in the tangle of their lives.
20836
post=154546
I just find myself drunk or baked one quaint evening with nothing to do so I play through what I got so far and see if it works.

Rinse, repeat.
I do this sometimes.

Well, not the baked part, but the drunk part.

(it also helps when you are facing writer's block)
Pages: first 12 next last