00:01Lots of people are excited by this new utility AI feature brought in, in state trees for UE 5.5

00:06So today, we're going to look at how we can use this utility AI feature to control the state of a, of an NPC

00:18So this character has four states

00:30He essentially, one, either explores the world he's in, two, if he sees us, he will chase us, three, if his stamina levels drop, he has to find a well to replenish them, and then at night time, he has to go to bed and stay there till dawn and then repeat his lifestyle

00:42The way that utility AI works is that it will calculate a score for each of those four states based on our, our sort of instructions, and Then it will choose the state with the highest score and execute it, and then at the end of that execution, it will go back and recalculate the scores for each of the available states, and thereby choose which state this character will be in

01:08Our setup is simple, so we've got a simple AI character with an AI running a state tree in a world where automatic navigation links have been set up

01:23I've done videos on this, and there is also a day-night cycle that is operating, and the day-night times will control whether he's awake or asleep, and the character has got an AI perception component on it, so it can detect us, and Choose to chase, and if you can't detect us, to explore

01:56So I've activated both of these state tree plugins and this day sequence plug-in

02:05Let's start by creating our first state tree, so if you right click, artificial intelligence, state tree, also be a state tree AI component, because we're going to put this on the AI controller, we'll click that, and we'll call that state tree utility, and then we can open it

02:32In our asset details, we can maybe set our schema, so our AI controller, minus AIC utility, so that's the one I've created

02:40The character is BP utility AI character, so we can set those up as our controller and context actor, and then in our states we can start adding

02:55So we'll add one first state, this will be our utility utility state action

03:13On this, we will then add some child states which you will be able to choose from

03:19So we'll say add child state, our first one will be explore world, our next one will be chase player, then we'll have a sort of go to sleep, and we'll have a fourth state which will be to replenish energy or thirst, so we'll just call that replenish

03:50I think I've just named it stamina, so I'll say replenish stamina

03:57Now basically, the way that utility AI works is if you click on your parent state and you go to the Selection behavior, instead of the select in order, there is an option which says, "Try to select children with highest utility," or you can select children weighted by utility

04:19We're going to choose with highest utility, so as it says, it will try to choose the state with the highest utility score

04:26If there is a tie, then it will choose them in order, so we'll set that there

04:34Then what you need to do is, when you add tasks to each of these states, you need to add a selection utility in order to calculate a weight

04:48So I've gone ahead and I've created the tasks for each of these states so that we can focus on how utility AI

04:57Will actually pick the states to go into, and this is all rather simple

05:06So to explore world first, we find a move to location, and then we move to it

05:11Then we delay

05:17Chase player, we find again the location of the player, move to it

05:22Go to sleep, we find the actor of a particular class, and then we move to it

05:29And replenish stamina, we look for an actor of a BP well type, move to it, spend some time there replenishing our stamina

05:37And just to help, I've set up also a global task so we can get our day night sequence from the world, and just one parameter, what are the bed that we need to go to when we go to sleep

05:46I've shown how you can

05:54Create tasks like this to get locations and move to them in previous videos, so we won't go into that today

06:03What we are interested in, though, is this utility, so how will it select the child with the highest utility? As we said, here there is a selection utility

06:10Now, there's two ways that you can do it

06:16One is by using variables that you can access in your state tree, and the other is by creating a consideration, which is a state tree utility consideration

06:23We'll try to do both today, and we'll start off with sort of accessing the variables via parameters that we've already set, so when it comes to explore world and The chase player tasks that will be based on whether or not the AI can detect us as a player, so in the AI controller, I've created an AI perception, and when it senses a player, it will set this player sensed variable, and it will send a state tree event using a gameplay tag, which I've set up some gameplay tags for this

07:14So if we go to our explore world first and under selection utility, click the plus, we've got an option of buil-in options of constant float and enum, we'll choose constant, we'll click that in the constant, instead of a float, we actually want a bool, so we'll say logic, and then we can compare one bool to Another, so we want to compare

07:47So the logic here is, if the AI controller cannot sense the player, he should explore

07:52So we drop down there, we look for our AI controller reference, and this is the player sense Boolean that we just created, and we'll say if that is equal to false, then this will return a score, and the scores are normalized, I think, between zero and one

08:12So this would return a score of one if the player was not sensed

08:23So if, um, player sensed equals false, this will return a score of one, and it will enter the explore world tasks

08:29For Chase player, we can use exactly the same logic, but we want it to be true, so

08:37Selection utility added consideration, we'll use a constant again, we'll say, "Oh, we want to do the logic," say, and that means one is equal to the other

08:55Again, we will bind this to our player sensed in our AI controller, and this time we will say if it is true, you return a score of one, in which case you will enter this state and execute these tasks

09:12So now we've set two selection utility score calculations just based on whether the play is sensed or not

09:19We can test this out, but before we do, I should explain why we've got on the explore world and the chase player events

09:27So if you go to the explore, I've added an on event, and if It receives the event state, Chase

09:33I've set up some gameplay tags: Chase, to Chase, and explore, replenish, and sleep

09:39It will then go back to the utility state to recalculate the score, and the reason is that once it's in a state, it will continue in that state until it gets the finished task, but in our perception, whenever the player is sensed changes, we are just sending a gameplay event, a state tree event, to trigger a change, and that change will force the tree to re-evaluate the score

10:13So if it's an explore and it gets the chase gameplay tag event, it will go back, re-evaluate the score, and enter Chase if it's in

10:24Chasing, and it loses sight of the player, it will get the explore gameplay event, and it will go back here, re-evaluate, and then it will go back into explore, so this should work for these two states, so let's test them out first

10:42Let's play, so he starts off in explore, as he runs, as he sees me, he starts the chasing, he'll come after us, and then he loses sight of us, he goes back to explore, so let's now create a score, utility score for go to sleep and replenish, and we'll be able to explain what this weight means when we do this, so if we go to the go to sleep task, which is just a task where it finds the bed's location and moves to it

11:15And stays there for a while before re-evaluating

11:20The way that we do that is, we've got our day sequence actor reference, and from there, we can get the time

11:31So I've done a video on how you can create a 24-hour day night sequence

11:43I've called mine BP Sun Moon, and basically, it sends out an event when the time is above 8:00 to say it's nighttime, and when the time is between 4:00 and after 4:00 in the morning, it says that it's no longer nighttime

11:54So we can bind to this nighttime Boolean in our state tree

12:00So in go to sleep, we can go to our selection utility

12:06Again, we can get another constant here, and we can say, again, put it as

12:11A Boolean, so a logic, and, and we want to say, is the left, so we will get our, from a global utility

12:18I've got a reference to the day actor sequence, and I can get the is nighttime Boolean from there, so is nighttime true, this will return a score of one, so he should then be going to bed

12:38If I go to replenish, and if we try to do that one now, so selection utility, the way this one works is that my, the AI character has got an, an attributes component

12:52The attributes component has got a stamina, and whenever the player is moving, the stamina reduces, and whenever he's in the replenished state, the stamina goes back up to 100, so we can

13:03Access this stamina from the character

13:11In order to do that, I've sort of created an internal variable and set it to the attributes, because otherwise you cannot gain access to it

13:22So from the replenish, we can get the selection utility, and here we want to get the flow float value

13:32Now this one is a bit more complicated, because when you go for the float, you get a response curve that you can create, or you can create your add your own external curve

13:39Then you get an input range, which you have to set, and what we wanted to do is that when the stamina is less than 40, in other words, when it's between 0 and 40, we want the AI

13:56character to go and replenish stamina or hydration at a well

14:02So under this, the curve, you can create your own points

14:08So if you click, if you press shift and click, and basically the curve goes between zero and one, I think it needs to be normalized

14:14So we'll say at zero, the stamina value is zero, we want the value to be zero

14:20Now we want the value to be one, we want this to return a score of one

14:25If I shift and click again, we then want to say if the value is 40, we want to return a value of one as well, so between 0 and 40

14:44However, this needs to be normalized, I think, for it to work, so that's where this input range comes in, our input

14:50Range for our stamina is between 0 and 100, so we can put that in there, which means that this can be normalized to 0.4

14:5840 would be 0.4, and at 40, we want to return a score of one

15:05We then want it to go from there and drop to zero, so shift and click again, and we can basically say if it's from 41 onwards, the value should be zero, so now we get this curve

15:18So from between 0 and 0.4, as in 0 to 40 of stamina, it will give us a score of one, and then it will drop down to zero from that point onwards, and we just make sure that it ends as well, so we can say at one, it's also zero, so that creates the curve

15:35This is the input range, the float

15:43Think you can, the input range will come in, you can put that as an interval now, so the input in, so this is where we bind to our stamina variable, so we can go to our actor, and from the attributes, there is a stamina value that we can bind to that

16:09Then I think in the input range, we need to make that an interval, so that's an interval between 0 and 100, compile that

16:22So this is the using a float value, and if the float value when normalized is between 0 and 40, 0 and 0.4, which is in the range 0 to 100 stamina, in other words, when it's less than 40% stamina, this will return a value of one

16:37Now the problem is obvious straight away is that

16:45They, they could all potentially return a value of one, in which case the state tree would just go to the first one that returns the highest value, so this is where the weight comes in, so you can prioritize which one should be more important, have a higher utility based on its weight

16:59So we could say that explore world has a weight of one, we want chase player to be more important than explore world, so we give it a weight of two, so if the scores were the same, it would go to chase player

17:17We want go to sleep to be more important than these two, so we want that to have a higher weight, because at night we want it to go, go to sleep, so we

17:22Give that a weight of three, and the replenish stamina, we might say that whenever its stamina or its hydration or whatever is low, it should always go and replenish regardless of anything, so we give that a weight of four, and we compile

17:42So if this works correctly, from the rout will come into the utility state action, the parent state, this state has got a selection behavior of select children with the highest utility

17:54On each one of the child states, there is a selection utility that we've added, and you can add multiple, so you could have two or three of these operating together, but our for the first two it just

18:01Looks at the AI controller player sensed variable

18:08If it's false, it says yes for explore world with a weight of one

18:15For Chase player, it says if this is sensed is true, it returns one, but with a weight of two

18:21In other words, it doubles it

18:27Go to sleep looks for the time in the day sequence actor, and it says if it is nighttime, then enter go to sleep with a weight of three, so it takes precedence over the previous two states

18:39And replenish, we've looked at this using a float value because we've got a stamina value between 0 to 100, where we've then created this curve, which is a normalized curve between zero and one, and Added the value, and the value that comes back is the score

19:00We've then bound that to the attribute stamina, told it the interval range is between 0 to 100, and we've given this a weight of four because we want it to take precedence over all of the previous states and have the highest utility score if it is valid

19:10So this should work now

19:19Let's go to our main level

19:29I said in our AI controller on the state tree, we've added our state tree utility that we're working on, so if we play, so he should end, start with exploring

19:37At some point, if he sees us, he should start to chase us

19:44He loses sight of us, he goes to explore again if his Stamina drops, so stamina's now dropped, so now he'll go looking for a well

19:52So he goes up to that well up there, and he will now replenish his stamina, so the bar goes up, up to 100

19:59And now, if the time is now past midnight, so now he's got the sleep, so he'll now go searching for his bed, and he will sleep there

20:17And then dawn comes, he sees us again, so he starts chasing and back to replenish

20:28So our state tree is using the utility score correctly to enter the correct state based on how we've told it to calculate that score, and this score, as I said, can be as complex as you like

20:40So you could have, we've just said like booleans here, but You could also add enums, constants, and have logic, ICal, and ORS to decide which one it enters

21:00A way of calculating a utility score is by using a consideration, so this is a blueprint state tree utility consideration, and this can then be used to calculate a more customizable utility score

21:21And you can create individual considerations for each score calculation, or you can combine them all into one and use some logic to identify the difference

21:32So we just create our first consideration, so state tree consideration blueprint base, and we'll just say this is our state tree consideration utility state, so once If you open that, you have the function override

21:47There is a description, which you can then allow it to show in, maybe just simply, you know, calculate score, and then there is the main function, which is the get score

22:02So this is the function that will return the score for this utility consideration, and generally, as we said, it's probably best to return a score between zero and one, so it's normalized

22:20So for instance, if we wanted to replace this with a calculate in our consideration, we probably need to have access to the AI controller, so we can go here, we can maybe create a variable, AI controller, we can set that as our

22:38AIC utility, this will be an input because we want it to be set

22:45Oh, actually, this can be a context, so it's automatically set

22:59And then if we replace this, so if we delete the one that we've just used and we add our consideration on the explore task, so state tree consideration, utility state, our context is set directly, the score will come back here multiplied by the weight

23:20So what we can do is we can get our AI controller and we could say get, get player sensed the variable, get player sensed, and then we could return a float based on that, so we could say select, we say based on that if the player is sensed, which is true, we

23:47Want to return zero, and if the player is not sensed, in this case, the not sensed for explore, we return turn one

23:59We could just put this into a function, so collapse that to a function and say, "get explore score." We could make that a pure function

24:11Now, when you do that, you might get an error

24:24It will say, "get explore spore score can modify state, cannot be called on self." So the way that you deal with that is go to the function, click on it, and make it a constant, and then that should then work

24:37Okay, so if it's not sensed, it will return one, and that will return one here, so we can go back there, and this should then work exactly the same

24:46So, for some reason, it crashed

24:56So what I've done is I've duplicated the state tree, and I've just removed all the old selection utilities we were using

25:06Sometimes you can change them and nothing happens, sometimes it crashes

25:16So we've added our first consideration to calculate a utility score, and this will send back the result of that with a weight of one

25:21We could create separate considerations to calculate a score for each of these states, or we can find a way of using the same consideration

25:34If we go back to our utility state, what we could do is we could create a state gameplay tag of the type

25:55Gameplay tag, and we could use that to differentiate what score we are looking for, so this could be our, just put that as default and expose it

26:15We could then, on the get score function, just take our state gameplay tag, get, do a switch, and then we can add pins to it, so we could add a pin case zero

26:29We could say this is our explore, so I said I've created gameplay tags state.explore, and that would then use this to return the value

26:45We could add another pin, and this could be state.chase

26:51We've also got replenish and sleep, so I'll go ahead and add those, so I've gone ahead and I've added all the different switch states

27:05Based on the gameplay tag, which we can send in, and then I've created the explore, which we saw, the chase score, which is exact opposite, and of course, making sure that you mark it as constant

27:18Then I've got the sleep score, which just needs to know, is it nighttime, and the replenish score, which just needs to know whether the stamina is less than 40, and I've exposed those, those variables that we need here

27:37So all of those will then set a final score value, which is a local variable, and we will send that final score back, so we can then use this one consideration to calculate the different scores for the different states

27:51Based on gameplay tags, so if we go back to our state in the explore world, we just need to make sure we set the tag

27:57So that is State explore, we don't need to set any of these for this particular state

28:05In the chase player, we can add the same state utility, and we just need to set the tag for Chase

28:12Again, it doesn't need those

28:20Go to sleep, on the other hand, we add our consideration here

28:27It needs to know, so it needs to know the tag, which is State sleep, and it needs to know, is it nighttime? So we can get that from our Global utility

28:40Our day sequence actor has an is nighttime Boolean, and the replenish, we can add the same

28:52Utility, we can add the state, and it needs to have the stamina, so we can get that from our actor, the attributes, and there's a stamina value, just like previously

29:10We'll leave the weights, so the explore has a weight of one, Chase has a weight of two, go to sleep weight of three, and replenish four, so that the normalized value coming through will be multiplied by the weight, so that we can prioritize which one takes precedence

29:32I've changed the state tree to the new one that we just created with the considerations, and it should work exactly the same, so he starts to explore, if he sees us, he begins to chase us, should keep Chasing us, and when his energy levels have dropped, he goes to replace, replenish, so he finds himself a well, jumps up there, replenishes his energy

30:05Now the clock says it's midnight, so now he should go and find a bed to sleep in, and he'll stay there till dawn, and then you come back and repeat the cycle

30:26So that's how you calculate a utility score using a consideration, and you can either use one consideration like we did to calculate all the scores for the different states, or you could create individual considerations, and again, you could add multiple of these considerations, and the scores would be added up

30:43To create a final score, you can mix and match, so you could, as long as, is that replenish stamina, get rid of the consideration and go back to using the one that we did previously

31:03So get a constant, actually get a float, create the response curve, put the input binding, the range, and you can then mix and match this approach with the considerations

31:29So I've added back the float with its response curve, like we did previously on the replenish stamina, and the rest are all using the consideration that we created

31:36Compile that, and again, it should work exactly the same

31:41It would explore, chase, and the replenish should work

31:53Again, so when he runs out of energy, he'll go and replenish and then go to sleep

32:10So that's a quick overview of how you use, um, Utility AI in your state trees with this selection behavior, creating your individual states under the parent state for it to consider, and then you can either create considerations like we did here and return a score, or you can use sort of the access to the inbuilt variables and access floats or Boolean and, and all kinds of kinds of logic that's available, intervals, maths, logic, and combine them

32:31So the key points to remember are that one, you need to set the behavior as select children

32:51With highest utility to it, it will then pick the state with the highest score, and you can weight these states so that you can decide which one should take priority if they have the same score

33:12The third thing to remember is that you can, is that once it enters a state, it doesn't check the score on tick, so it's only when it returns back to here will it check it again, so you can force it to recheck by using state tree events, so it goes back and finds another state

33:34Otherwise, the rest of it, it's all quite simple

33:41You can sort of combine different ways of calculating the utility, which can be simple or as complex as you like, and Overall, this is going to be very useful if you're creating complex behavior, but just want to do so with a simple state tree setup, so I hope that helps explain it.