Now that I have my projects set up, let's look at the test itself. This is a test that you can find on our support site under GameDriver support. It's called 'Testing the 2D Game Kit'. There are two tutorials available. The first is a basic tutorial that walks through how to define this test and explains what's being done at every step. The second tutorial is for a specific version, which is called the 'Corrected Version of the 2D Game Kit'. I won't delve into every specific line of this, but I will give you a sense of how this test is structured, what types of inputs are being performed, and what checks are being made to verify that things are happening as they should.
First, as I mentioned earlier, we have our class definition. This is where we provide all our global settings and values that will be used throughout the test. Before we do anything else, we need to initialize some things. For example, we have our static string path in this case, which I could then override using inputs if I'm running this from a command line. I've set some default values here, just as examples.
Here, I am setting the mode in which I'm running the test. If I want to run this in the editor, or if I want to run this on a standalone Windows or MacOS executable, or on a mobile device, I might change these values to accommodate that.
Then, I have my API client. This is the method we use; we're instantiating this class and now we're using this to send commands to the agent directly. Within the one-time setup, this is where we perform our connect function. We're going to use some logic here, based on the values above, to determine how I'm going to execute this test. Am I going to run this in the Unity editor, or am I going to run this on a standalone?
For example, if the path to the executable is not null, then I'm taking the path of the executable and I'm passing it to a static method, which is API client.launch. This method uses the path to the executable, be it a Windows path, a MacOS path, etc. Linux is also supported. It will actually start a process to run that executable. The next command after that is to connect to the agent. These are separate because the static method is launching without using the API that we instantiated above.
Then we have 'Connect', which is going to connect to a host. It could be local, it could be remote, it could be a machine running somewhere else on my network that I'm going to broadcast a start signal to in order to initialize the editor into play mode and connect to it. All of these options are supported and they're all covered in our API documentation, which is available on GitHub. Some of the links will be shown at the end of the presentation.
Next, we have the port number. We're connecting on port 19734. If you have multiple clients running on the same machine or multiple agents, we might set different port numbers for each of them. You can change this using functions within the Unity project or within the agent to ensure that you have a unique port for each of the agents that are running.
The third argument is whether or not we are starting the play mode in the editor, using a true or false Boolean. In this case, if my path to the executable is not null, I'm assuming that we're not starting an editor, and therefore the autoplay is set to false. The last argument is just a timeout. As we progress, I'll provide less detail because you'll start to understand as I explain each of these.
Otherwise, if the test mode equals 'IDE', you can call these whatever you like. We're connecting to the localhost again, but this time, we're going to use autoplay to start the editor in play mode. Now, I don't need to press play on my test and then switch over to my editor to press play or start the game. This function will do it for me automatically.
The last case is assuming nothing else, that there's a version of the game running somewhere. It could be connected to the computer using ADB on Android for testing on Android devices. It could be in XCode, or it could be that I already have an executable that's running at a specific place in the game where I want to start my test. I'm just going to connect, and I'm not starting anything.
The next few lines are where we enable our hooking, which is how we provide input to the game during replay. In this case, we know that the game is using keyboard and mouse inputs. I've selected these individually because I know there are no XR inputs or gamepad inputs. It wouldn't make sense to provide all of that in the same method. But you can also use 'hookingObjects.All' to enable all hooking. You can see some of the options available, such as XR legacy, mouse, keyboard, gamepad, etc.
Lastly, before we start our tests, the last thing that we want to do is navigate past the start menu. This could likely be a separate test in itself to ensure that the start menu is working or that we're registering or logging on as a user. But in this case, the game is very simple. The start menu is the only thing getting in the way for me to start the test. I've added this to the one-time setup function, and all the tests below this will run from that assumed position that we've started the game.
In this case, we're waiting for the start menu. We use 'WaitForObject' and we're waiting for the start button to appear. If I go back to my scene for a moment, that's this button right here. If I didn't know the path to it, perhaps if it was nested or buried somewhere in the scene (which is sometimes the case), then I may take this object and right-click it and say, 'Give me the path to this object.' You'll see in the console that it provides the relative path, which is exactly what we see here.
This is "//start" and then "@name equals start button". Once this returns true, we will click on that object and we're just going to click with MouseButtons.LEFT. Again, we find that object using its Hierarchy Path and then perform the click. An alternative to this might be that we perform a click just using the raw click function at certain coordinates. However, as games can change their form factor, and the position of objects can shift based on their form factor, whether it's a mobile device, tablet, or PC, you do have the capability of clicking on a raw X-Y coordinate for that object. Another method might be that we search for the position of the object first using getObjectPosition. Store that value into a Vector2, and then use that as the input for my click coordinates. These are just some of the ways you can approach the same functionality here for different scenarios.
[00:14:56] Once we get past that, we're actually going to start running our test. As you can see here by the test attribute, we've got the test and then order zero, meaning that this is the first test that's going to run. We're calling this one zone one, and then we're waiting for the "Ellen" object to appear. Now, I know from experience with this game that this is the name of the character, but if I didn't know that, I could find that information simply by coming over here. And I see a little asterisk next to the start since I'm going to press save. So I'm going to come in here and I've started my project and you can see, I can click around and stuff. There's the whole scene with a bunch of different objects and information that's available to me as the tester. In this case, it just makes sense that Ellen is the character. We know that from playing the game that when I move around in the scene, this is the character and we can see her moving through the transform object that's available here. If we look through this object, we might see the player controller, the two rigid bodies, the scripts that are attached to her, the different methods for input, etc. So this tells me as a tester, this is generally what I'm looking for in terms of defining my test steps.
We're waiting for the Ellen object to appear, and then we are navigating to or looking for these InfoPost objects. You can see api.GetObjectPosition. We're storing this into a Vector3 for the first InfoPost, or InfoPost zero. The InfoPosts, if you go back to the scene here, there are two of them. And when I hover over one of these with the player character, the dialog box appears. This is the tutorial part of the game. So this is functionality that I probably want to test as I'm going through and building my test for this project. So when I find the InfoPost in the scene here, I can look through all of the properties and values. But really, what I'm looking for is whether or not this object appears when I move the character to that position as it does here in this example. I hover the character over the post, and it appears. So I want to check if that property exists when we move the character to that position. So in order to do that, I'm going to store the value in a Vector3, as I mentioned, and I'm going to set the position of the player, which is the Ellen character again. Ellen has a component called the UnityEngine.Transform, and we're going to find the position value or the position property and set that to that of the InfoPost. You can see where I saved it. Here is the input InfoPost zero. I'm setting the ObjectFieldValue of the Ellen character to that of the InfoPost, which is just going to directly place Ellen on top of that. And then the next test or the next step is the test, I should say, in where I'm doing an assertion. Remember, those are actual tests where we are waiting for the object value of the dialog canvas to turn into active. This is where the InfoPost pops up a message. If I look for the dialog canvas, for example, come back here, and search for dialog canvas. This is where I would be able to test. Did this object appear? And then I'm going to move to the next one. Basically finding the next InfoPost and calling it InfoPost1. You can see there is an instance predicate here, which is one. This tells me that it's the second InfoPost in the scene, the first being zero or no value. And then we're setting the position of the character again to that of InfoPost1. And then checking, did the dialog canvas appear? So did the object dialog canvas appear active? And if not, then we failed the test.
Now, the next part of this is providing some input because as we've moved through the scene, the second InfoPost says, "Weird, it feels spongy and strange underfoot. What happens if we try S and space at the same time?" Which means when I press S and space, I should go down to the next level. That's the first change from one scene to the next. We're going to perform those inputs. We're going to press S and we're going to press space. Now, we could do this in one line, pressing S as a modifier and then space to go down. But there is sort of a timing involved in this.
We found that holding the space bar down for about half a second allows us to move downward. Now, the length of time that we press these keys really does matter, and I'll explain why a little bit later. We have the key press function or command which performs an array of keys or inputs, an array of keys followed by a duration. Now, if I delete this for a second, you'll see that the number of frames is the next argument in this command. The number of frames is a tricky property or argument for any sort of input. Imagine, if you will, that I'm running this on a high-end machine with superior CPU memory, graphics processing units, etc. I would achieve a much higher frames per second rate than if I was to run this on my mobile device. To compensate for this, we use a trick, instead of providing the raw value of 100, for example, which is a perfectly valid figure for the number of frames, we're going to check the current running frame rate of the engine. We do this using the GetLastFrames per second command in the agent, which precisely informs us of the current frames per second. Then, we're going to use that as a measure for how many seconds we want to run this command.
This machine that I'm running the test on achieves somewhere in the range of a few hundred frames per second. Stats here will show us exactly how. A couple of hundred frames per second. So one second will be approximately 200 frames. However, on my phone, the frame rates are locked at around 90. This value will, therefore, be different. Instead of providing complex logic to determine where I'm running it, I just use this approach of capturing the last frames per second, and then I can use that as a one-second input, or multiply it by two, divide it by two, etc. This will give me roughly the number of frames of input that I'm looking for. We do support frame-perfect inputs, as well as this approach where you can kind of use an analogy for how long you want to press the keys.
So after that long-winded explanation, we perform the key presses, and then we wait for another object to appear. This is going to tell us that we are in the next scene. In the next scene, we're looking for an object, which is a key. It has a component that is active and enabled equals true. Let me explain what that means. There are actually four keys in this scene. One is here, the one that I'm looking for. The other three are in the corner. In a lot of games, developers use prefabs and clone them all over the scene. You might have a tennis game with 100 balls scattered all over the court and one that's being used back and forth between the players, but they're all essentially named 'ball'. In this case, we have four objects named 'Key'. If I search for 'Key' here, I'll find four keys and three key icons.
So the key that I want is the only one that is active and enabled equals true, which is right here. When I perform this search, I'm looking for a tag, which is 'objective', and the name equals 'key'. Using that Boolean operator, we're searching for one of those objects that has the UnityEngine.Behavior with a property of ActiveAndEnabled equals true. This is probably something we could have easily done by saying, give me the key with the field of active. There are actually several ways that you can check for objects being active in the scene, but we wanted to demonstrate what it looks like to combine elements in a search like this. And then we check that we're in the right zone.
Are we in Zone2 and using GetSceneName? The expected value is Zone2, and the actual value is queried from the scene to ask, 'What is the scene name?' If these two values don't line up, the test fails. Finally, before we finish, we capture a screenshot. This provides a snapshot of the screen, which is stored in our test folder when running from a command line. Otherwise, it serves as a temp directory. We now have a screenshot showing exactly what was on the screen when we moved from Scene One to Scene Two.
Next, we process Scene Two. I won't step through each command, but essentially, we are locating the position of the key and setting the player to that key position again, similar to what we did with InfoPost. We then check if the key has entered our inventory. We use a GetObjectFieldValue of a color type to verify if the key has changed color. Remember, there are four keys here; the one below, which is active and rotating, and the three above, which are the key icons. We check if the key icon clone zero, which is the first one, changes from black to color. The transparency value on that key changes from 0 to 1. This indicates that we have our key. The next step sets the Ellen character's position to a specific value. We set a new Vector3 value right near the scene's edge, so that when we press the next key to the right, it takes us to the next scene, Zone3. Each step along the way confirms that we are progressing through the puzzle-solving process. In the next level, we move to the weapon pickup. This is necessary to verify that we can use the weapon. We then get another key, check the color again of the next key in the UI using Key Clone 1, and then move to the next scene, and so on. This process repeats through all five levels of the game, capturing all the keys, acquiring all the prerequisites, and finally bringing us to the Boss, which we won't fight in this test. The aim here is to ensure that everything is functioning correctly so that a tester who wishes to play the boss level doesn't waste time getting there.
If I were to play this game, it would probably take me 12 to 15 minutes to finish, if I had become proficient at it. However, doing so would be repetitive and not very valuable. Using this testing approach, I can reduce the time required and ensure all prerequisites are functioning before I fight the Boss. A common scenario in game development is that developers provide testers with a shortcut to the final boss of the scene and ask them to evaluate its difficulty level. But this shortcut bypasses all the prerequisites. If something were to break along the way, and we've only tested it once at the very beginning, we wouldn't know if anything had changed or broken. Now, let's run this test and make sure everything is set up properly. My path is Null, my test mode is in the IDE, and I know I'll be connecting to the editor here. I run this test through the Test Explorer in Visual Studio. Before I do that, I'm going to clean my project to ensure I have the most up-to-date version of the DLLs or assemblies from the project, in case I updated those. There are five tests defined here in the Test Explorer, and I'm just going to hit 'run'.
Now, it's hands-off keyboard time because this test will connect to the editor, enter play mode, and start running all the tests we've created. The overall process of defining this test probably took less than 2 hours. The initial challenge is identifying the inputs, objects, and interactions when performing certain activities. Once you have that figured out, the rest becomes easy. Then it's simply a matter of deciding what you want to check in this test. Do you want to verify that the text value in the InfoPost is correct? Or are you validating a translation from one language to another? In that case, we might feed data from a text file specifying the InfoPost and the information in the target language.
[00:28:42] There's no limit to what we could add once we understand how the game is implemented. Now, in about 40 seconds, this test ran. We'll verify shortly that I went through all the prerequisites, grabbed the three keys, enabled the weapon for attacks, used that weapon to break a wall to access the third key, and finally reached the Boss. The test may have been too quick to catch all these actions. The important thing here is that the connection count is now zero. This indicates that I'm now in control as a user and ready to fight the Boss. This is the challenging part, knowing that everything prior to this is working. I've tested all the important functions in the gameplay loop without compromising quality or bypassing anything. Importantly, the developer doesn't need to implement anything specifically for the tester that would need to be removed later, which could be seen as a risk if they remove functionality. You may not be testing the same thing that is being delivered to users. This Boss is surprisingly difficult. I'm going to go get some help. By not having to add any specific code to facilitate these tests, I'm assured that things are working as they should be. Alright, let's not do that because I'll probably fail. Oh, spikes! Now that our test has run, what have we learned from the 2D Game Kit example?
Well, we know that we want to isolate features and test them individually for greater stability. We want to separate our tests from the test setup to the inputs we're providing, the actions we're performing, and the expected outcomes. We achieve this by handling inputs, menus, and the gameplay loop separately. We provide our inputs, perform a check, open a menu, run a check, do something in the gameplay loop, and ensure there's an assertion there to check the expected outcome. We don't want too many actions happening in a single test. It's about setting up, inputting actions, and checking results. Now, let's take a look at a more complicated example with the 3D Game Kit.